[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:11 dist-test-slave-zqpb krb5kdc[19916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:11 dist-test-slave-zqpb krb5kdc[19916](info): set up 2 sockets
Jun 19 02:45:11 dist-test-slave-zqpb krb5kdc[19916](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 02:45:12.431278 19909 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:12 dist-test-slave-zqpb krb5kdc[19916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:12.556082 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:35383
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:46521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:35383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:12.637049 19929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:12.637264 19929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:12.637293 19929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:12.638479 19929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:12.638525 19929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:12.638542 19929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:12.638558 19929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:12.638573 19929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:12.640144 19929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:35383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:35383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:12.640403 19929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:12.640653 19929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:12.645767 19934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:12.645876 19929 server_base.cc:1034] running on GCE node
W20260619 02:45:12.645757 19935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:12.645861 19937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:12.646226 19929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:12.646508 19929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:12.647655 19929 hybrid_clock.cc:648] HybridClock initialized: now 1781837112647640 us; error 30 us; skew 500 ppm
Jun 19 02:45:12 dist-test-slave-zqpb krb5kdc[19916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837112, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:12.650102 19929 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:12.650555 19929 webserver.cc:460] Webserver started at http://127.19.113.126:39695/ using document root <none> and password file <none>
I20260619 02:45:12.650766 19929 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:12.650811 19929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:12.650952 19929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:12.652031 19929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "e3361b274aac4744845af4e06cf9953a"
format_stamp: "Formatted at 2026-06-19 02:45:12 on dist-test-slave-zqpb"
server_key: "8f6cd4a13d1167abc1b50d053e070d93"
server_key_iv: "54e02236e10cae6e5b4e5ab6dfc0d290"
server_key_version: "encryptionkey@0"
I20260619 02:45:12.652369 19929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "e3361b274aac4744845af4e06cf9953a"
format_stamp: "Formatted at 2026-06-19 02:45:12 on dist-test-slave-zqpb"
server_key: "8f6cd4a13d1167abc1b50d053e070d93"
server_key_iv: "54e02236e10cae6e5b4e5ab6dfc0d290"
server_key_version: "encryptionkey@0"
I20260619 02:45:12.653730 19929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:12.654476 19944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:12.654629 19929 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:12.654712 19929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "e3361b274aac4744845af4e06cf9953a"
format_stamp: "Formatted at 2026-06-19 02:45:12 on dist-test-slave-zqpb"
server_key: "8f6cd4a13d1167abc1b50d053e070d93"
server_key_iv: "54e02236e10cae6e5b4e5ab6dfc0d290"
server_key_version: "encryptionkey@0"
I20260619 02:45:12.654773 19929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:12.679234 19929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:12.681771 19929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:12.681937 19929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:12.686147 19929 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:35383
I20260619 02:45:12.686193 19996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:35383 every 8 connection(s)
I20260619 02:45:12.686623 19929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:12.687624 19997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:12.690034 19997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a: Bootstrap starting.
I20260619 02:45:12.690625 19997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:12.690927 19997 log.cc:824] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:12.691134 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 19929
I20260619 02:45:12.691260 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:12.691692 19997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a: No bootstrap required, opened a new log
I20260619 02:45:12.692008 19909 external_mini_cluster.cc:1428] Setting key a546fe8b173b4d81eb9f272f142d27b9
I20260619 02:45:12.694000 19997 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3361b274aac4744845af4e06cf9953a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35383 } }
I20260619 02:45:12.694151 19997 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:12.694186 19997 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3361b274aac4744845af4e06cf9953a, State: Initialized, Role: FOLLOWER
I20260619 02:45:12.694329 19997 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3361b274aac4744845af4e06cf9953a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35383 } }
I20260619 02:45:12.694410 19997 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:12.694442 19997 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:12.694489 19997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:12.695425 19997 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3361b274aac4744845af4e06cf9953a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35383 } }
I20260619 02:45:12.695528 19997 leader_election.cc:302] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3361b274aac4744845af4e06cf9953a; no voters:
I20260619 02:45:12.695686 19997 leader_election.cc:288] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:12.695775 20002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:12.695930 20002 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [term 1 LEADER]: Becoming Leader. State: Replica: e3361b274aac4744845af4e06cf9953a, State: Running, Role: LEADER
I20260619 02:45:12.695955 19997 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:12.696626 20002 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3361b274aac4744845af4e06cf9953a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35383 } }
I20260619 02:45:12.697706 20004 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3361b274aac4744845af4e06cf9953a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3361b274aac4744845af4e06cf9953a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35383 } } }
I20260619 02:45:12.698105 20004 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [sys.catalog]: This master's current role is: LEADER
W20260619 02:45:12.698606 20016 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:12.698663 20016 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:45:12.698930 20017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:12.699128 20017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:12.697898 20003 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3361b274aac4744845af4e06cf9953a. Latest consensus state: current_term: 1 leader_uuid: "e3361b274aac4744845af4e06cf9953a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3361b274aac4744845af4e06cf9953a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35383 } } }
I20260619 02:45:12.699753 20003 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:12.700820 20017 catalog_manager.cc:1347] Generated new cluster ID: d9a4750d5e504ee99db6d37e89139950
I20260619 02:45:12.700876 20017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:12 dist-test-slave-zqpb krb5kdc[19916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:12.710405 20017 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:12.711174 20017 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:12.712458 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:12.693247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56806 (local address 127.19.113.126:35383)
0619 02:45:12.693485 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:12.693496 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:12.693741 (+ 245us) server_negotiation.cc:407] Connection header received
0619 02:45:12.696190 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:12.696198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:12.696258 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:12.696394 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:12.698906 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:12.700014 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:12.701425 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:12.702048 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:12.702168 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:12.707443 (+ 5275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:12.707449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:12.707450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:12.707464 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:12.709481 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:12.710422 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:12.710425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:12.710426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:12.710553 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:12.710941 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:12.710942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:12.710943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:12.711225 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:12.711384 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:12.711842 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 02:45:12.712193 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
I20260619 02:45:12.714933 20017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3361b274aac4744845af4e06cf9953a: Generated new TSK 0
I20260619 02:45:12.715102 20017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:12.752297 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35383
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:46521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:12.840309 20023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:12.840548 20023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:12.840605 20023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:12.841866 20023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:12.841905 20023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:12.841943 20023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:12.843492 20023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:12.843739 20023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:12.844007 20023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:12.849089 20029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:12.849069 20031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:12.849123 20028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:12.849272 20023 server_base.cc:1034] running on GCE node
I20260619 02:45:12.849506 20023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:12.849763 20023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:12.850925 20023 hybrid_clock.cc:648] HybridClock initialized: now 1781837112850894 us; error 90 us; skew 500 ppm
Jun 19 02:45:12 dist-test-slave-zqpb krb5kdc[19916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837112, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:12.857296 20023 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:12.857836 20023 webserver.cc:460] Webserver started at http://127.19.113.65:40115/ using document root <none> and password file <none>
I20260619 02:45:12.858084 20023 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:12.858137 20023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:12.858271 20023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:12.859457 20023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "c3b57e58b32f45b692530bd798f836f0"
format_stamp: "Formatted at 2026-06-19 02:45:12 on dist-test-slave-zqpb"
server_key: "2092cf44872a44a57f00062c2b77d728"
server_key_iv: "2ff611b6663ab738323dd9095b68170c"
server_key_version: "encryptionkey@0"
I20260619 02:45:12.859812 20023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "c3b57e58b32f45b692530bd798f836f0"
format_stamp: "Formatted at 2026-06-19 02:45:12 on dist-test-slave-zqpb"
server_key: "2092cf44872a44a57f00062c2b77d728"
server_key_iv: "2ff611b6663ab738323dd9095b68170c"
server_key_version: "encryptionkey@0"
I20260619 02:45:12.861301 20023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:12.862152 20038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:12.862510 20023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:12.862815 20023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "c3b57e58b32f45b692530bd798f836f0"
format_stamp: "Formatted at 2026-06-19 02:45:12 on dist-test-slave-zqpb"
server_key: "2092cf44872a44a57f00062c2b77d728"
server_key_iv: "2ff611b6663ab738323dd9095b68170c"
server_key_version: "encryptionkey@0"
I20260619 02:45:12.863073 20023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:12.873867 20023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:12.876633 20023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:12.876784 20023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:12.877027 20023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:12.877355 20023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:12.877389 20023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:12.877420 20023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:12.877439 20023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:12.883137 20023 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:41493
I20260619 02:45:12.883198 20151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:41493 every 8 connection(s)
I20260619 02:45:12.883507 20023 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:12.888513 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 20023
I20260619 02:45:12.888624 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:12.888787 19909 external_mini_cluster.cc:1428] Setting key 0ab8e56ead006e8f552a2c06015dfd02
Jun 19 02:45:12 dist-test-slave-zqpb krb5kdc[19916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837112, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:12.893669 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:12.884292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:58703 (local address 127.19.113.126:35383)
0619 02:45:12.884387 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:12.884390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:12.884570 (+ 180us) server_negotiation.cc:407] Connection header received
0619 02:45:12.885133 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:12.885135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:12.885147 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:12.885196 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:12.886231 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:12.886764 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:12.887802 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:12.887998 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:12.888058 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:12.891063 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:12.891065 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:12.891065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:12.891072 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:12.892349 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:12.892756 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:12.892757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:12.892758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:12.892813 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:12.893039 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:12.893040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:12.893040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:12.893156 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:12.893212 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:12.893480 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 02:45:12.893573 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:45:12.893862 20154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:12.884223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:35383 (local address 127.19.113.65:58703)
0619 02:45:12.884437 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:12.884480 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:12.884997 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:12.885275 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:12.885280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:12.885908 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:12.886129 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:12.886135 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:12.886854 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:12.886855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:12.887713 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:12.887716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:12.888072 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:12.888073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:12.888199 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:12.888891 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:12.888902 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:12.890954 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:12.892430 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:12.892437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:12.892440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:12.892683 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:12.892868 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:12.892869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:12.892869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:12.892977 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:12.893212 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:12.893219 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:12.893363 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:45:12.893452 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:45:12.893641 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":94,"threads_started":1}
I20260619 02:45:12.894343 20152 heartbeater.cc:347] Connected to a master server at 127.19.113.126:35383
I20260619 02:45:12.894438 20152 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:12.894630 20152 heartbeater.cc:511] Master 127.19.113.126:35383 requested a full tablet report, sending...
I20260619 02:45:12.895255 19961 ts_manager.cc:194] Registered new tserver with Master: c3b57e58b32f45b692530bd798f836f0 (127.19.113.65:41493)
I20260619 02:45:12.896204 19961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:58703
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:12.919298 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35383
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:46521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:13.021775 20157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:13.022001 20157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:13.022027 20157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:13.023203 20157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:13.023242 20157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:13.023270 20157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:13.024710 20157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:13.024894 20157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:13.025084 20157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:45:13.030779 20157 server_base.cc:1034] running on GCE node
W20260619 02:45:13.030756 20165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:13.030759 20163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:13.030752 20162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:13.031143 20157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:13.031397 20157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:13.032588 20157 hybrid_clock.cc:648] HybridClock initialized: now 1781837113032570 us; error 32 us; skew 500 ppm
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[19916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:13.035269 20157 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:13.035754 20157 webserver.cc:460] Webserver started at http://127.19.113.66:46475/ using document root <none> and password file <none>
I20260619 02:45:13.035980 20157 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:13.036026 20157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:13.036135 20157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:13.037242 20157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "617c246c149e46a2b3031eadf123d6d4"
format_stamp: "Formatted at 2026-06-19 02:45:13 on dist-test-slave-zqpb"
server_key: "8260d086dec3b1dec9e4ba83c19aef2d"
server_key_iv: "f9f245de543b35965a0a29b3164d4ed9"
server_key_version: "encryptionkey@0"
I20260619 02:45:13.037587 20157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "617c246c149e46a2b3031eadf123d6d4"
format_stamp: "Formatted at 2026-06-19 02:45:13 on dist-test-slave-zqpb"
server_key: "8260d086dec3b1dec9e4ba83c19aef2d"
server_key_iv: "f9f245de543b35965a0a29b3164d4ed9"
server_key_version: "encryptionkey@0"
I20260619 02:45:13.038985 20157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:13.039754 20172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:13.039924 20157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:13.040009 20157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "617c246c149e46a2b3031eadf123d6d4"
format_stamp: "Formatted at 2026-06-19 02:45:13 on dist-test-slave-zqpb"
server_key: "8260d086dec3b1dec9e4ba83c19aef2d"
server_key_iv: "f9f245de543b35965a0a29b3164d4ed9"
server_key_version: "encryptionkey@0"
I20260619 02:45:13.040079 20157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:13.057235 20157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:13.059753 20157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:13.059895 20157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:13.060187 20157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:13.060506 20157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:13.060549 20157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:13.060571 20157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:13.060583 20157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:13.067890 20157 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:38539
I20260619 02:45:13.068013 20285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:38539 every 8 connection(s)
I20260619 02:45:13.068279 20157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:13.075364 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 20157
I20260619 02:45:13.075449 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:13.075656 19909 external_mini_cluster.cc:1428] Setting key a84afaacf4e99bf4e3ce90a9ebb0c507
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[19916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:13.079746 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:13.069517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:44343 (local address 127.19.113.126:35383)
0619 02:45:13.069589 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:13.069591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:13.069642 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:45:13.070267 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:13.070268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:13.070282 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:13.070384 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:13.071714 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.072249 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.073296 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.073486 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.073548 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:13.076477 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:13.076479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:13.076479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:13.076487 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:13.078116 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:13.078634 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:13.078635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:13.078635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:13.078718 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:13.079012 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:13.079013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:13.079014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:13.079161 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:13.079281 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:13.079556 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 02:45:13.079653 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:45:13.079833 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:13.069344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:35383 (local address 127.19.113.66:44343)
0619 02:45:13.069516 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:13.069561 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:13.070070 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:13.070520 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:13.070525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:13.071322 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:13.071547 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:13.071553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.072364 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.072366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:13.073192 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:13.073195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.073590 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.073591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:13.073940 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:13.074487 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:13.074496 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:13.076398 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:13.078216 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:13.078224 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:13.078226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:13.078554 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:13.078784 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:13.078784 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:13.078785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:13.078941 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:13.079283 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:13.079289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:13.079421 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:45:13.079506 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:45:13.079646 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260619 02:45:13.080436 20286 heartbeater.cc:347] Connected to a master server at 127.19.113.126:35383
I20260619 02:45:13.080549 20286 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:13.080724 20286 heartbeater.cc:511] Master 127.19.113.126:35383 requested a full tablet report, sending...
I20260619 02:45:13.081261 19961 ts_manager.cc:194] Registered new tserver with Master: 617c246c149e46a2b3031eadf123d6d4 (127.19.113.66:38539)
I20260619 02:45:13.081785 19961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:44343
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:13.108099 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35383
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:46521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:13.193516 20291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:13.193760 20291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:13.193799 20291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:13.195111 20291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:13.195163 20291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:13.195212 20291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:13.196950 20291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:13.197240 20291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:13.197469 20291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:13.203070 20299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:13.203075 20296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:13.203266 20291 server_base.cc:1034] running on GCE node
W20260619 02:45:13.203118 20297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:13.203609 20291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:13.203876 20291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:13.205056 20291 hybrid_clock.cc:648] HybridClock initialized: now 1781837113205039 us; error 36 us; skew 500 ppm
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[19916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:13.207859 20291 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:13.208380 20291 webserver.cc:460] Webserver started at http://127.19.113.67:40627/ using document root <none> and password file <none>
I20260619 02:45:13.208590 20291 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:13.208642 20291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:13.208773 20291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:13.209993 20291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "d6201c96d2294a8a946c6b9852af10aa"
format_stamp: "Formatted at 2026-06-19 02:45:13 on dist-test-slave-zqpb"
server_key: "c195086d80b813f6066081980f82e1e9"
server_key_iv: "a1944eaf7672463979d863cfe6f73739"
server_key_version: "encryptionkey@0"
I20260619 02:45:13.210353 20291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "d6201c96d2294a8a946c6b9852af10aa"
format_stamp: "Formatted at 2026-06-19 02:45:13 on dist-test-slave-zqpb"
server_key: "c195086d80b813f6066081980f82e1e9"
server_key_iv: "a1944eaf7672463979d863cfe6f73739"
server_key_version: "encryptionkey@0"
I20260619 02:45:13.211781 20291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:13.212652 20306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:13.212845 20291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:13.212935 20291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "d6201c96d2294a8a946c6b9852af10aa"
format_stamp: "Formatted at 2026-06-19 02:45:13 on dist-test-slave-zqpb"
server_key: "c195086d80b813f6066081980f82e1e9"
server_key_iv: "a1944eaf7672463979d863cfe6f73739"
server_key_version: "encryptionkey@0"
I20260619 02:45:13.212996 20291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:13.223042 20291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:13.225276 20291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:13.225420 20291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:13.225648 20291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:13.225946 20291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:13.225987 20291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:13.226019 20291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:13.226042 20291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:13.231985 20291 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:36131
I20260619 02:45:13.232028 20419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:36131 every 8 connection(s)
I20260619 02:45:13.232345 20291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:13.233719 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 20291
I20260619 02:45:13.233794 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:13.233928 19909 external_mini_cluster.cc:1428] Setting key ebbf2247aa9239dc2c4aabb225a8cbc3
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[19916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:13.241561 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:13.233110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:43545 (local address 127.19.113.126:35383)
0619 02:45:13.233190 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:13.233193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:13.233429 (+ 236us) server_negotiation.cc:407] Connection header received
0619 02:45:13.234127 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:13.234128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:13.234142 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:13.234193 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:13.235288 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.235791 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.236768 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.236957 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.237012 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:13.239191 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:13.239193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:13.239193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:13.239200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:13.240294 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:13.240681 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:13.240682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:13.240682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:13.240735 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:13.240949 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:13.240950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:13.240951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:13.241060 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:13.241114 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:13.241369 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:45:13.241458 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:45:13.241725 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:13.233109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:35383 (local address 127.19.113.67:43545)
0619 02:45:13.233283 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:13.233321 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:13.234009 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:13.234266 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:13.234270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:13.234991 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:13.235203 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:13.235208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.235873 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.235874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:13.236686 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:13.236688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.237033 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.237034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:13.237166 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:13.237626 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:13.237634 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:13.239104 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:13.240369 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:13.240375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:13.240378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:13.240613 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:13.240790 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:13.240790 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:13.240791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:13.240897 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:13.241114 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:13.241120 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:13.241257 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 02:45:13.241345 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:45:13.241509 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260619 02:45:13.242116 20420 heartbeater.cc:347] Connected to a master server at 127.19.113.126:35383
I20260619 02:45:13.242215 20420 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:13.242405 20420 heartbeater.cc:511] Master 127.19.113.126:35383 requested a full tablet report, sending...
I20260619 02:45:13.242753 19961 ts_manager.cc:194] Registered new tserver with Master: d6201c96d2294a8a946c6b9852af10aa (127.19.113.67:36131)
I20260619 02:45:13.243170 19961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:43545
I20260619 02:45:13.245730 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[19916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837113, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[19916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837113, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:13.267975 20427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:13.260483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.19.113.65:41493)
0619 02:45:13.260657 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:13.260661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:13.260672 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:13.260697 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:13.260699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:13.260713 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:13.260774 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:13.261361 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.262124 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.262811 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.263219 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.263275 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:13.265427 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:13.265435 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:13.265435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:13.265442 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:13.266992 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:13.267332 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:13.267335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:13.267335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:13.267392 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:13.267543 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:13.267544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:13.267544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:13.267671 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:13.267804 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:13.267837 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 02:45:13.267898 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
W20260619 02:45:13.268333 20066 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41446
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[19916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[19916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:13.289995 20427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:13.282846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.19.113.65:41493)
0619 02:45:13.282926 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:13.282928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:13.282992 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:45:13.283072 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:13.283073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:13.283084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:13.283181 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:13.283742 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.284238 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.284941 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:13.285149 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:13.285208 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:13.287540 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:13.287542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:13.287542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:13.287550 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:13.288947 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:13.289289 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:13.289290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:13.289291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:13.289345 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:13.289503 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:13.289504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:13.289504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:13.289611 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:13.289711 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:13.289804 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:45:13.289888 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:45:13.290719 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 20023
I20260619 02:45:13.295024 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 20157
I20260619 02:45:13.299309 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 20291
I20260619 02:45:13.304226 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 19929
2026-06-19T02:45:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1906 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[20434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[20434](info): set up 2 sockets
Jun 19 02:45:13 dist-test-slave-zqpb krb5kdc[20434](info): commencing operation
krb5kdc: starting...
W20260619 02:45:14.334450 19909 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:14 dist-test-slave-zqpb krb5kdc[20434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:14.477856 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:35993
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:36037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:35993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:14.563812 20447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:14.564036 20447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:14.564060 20447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:14.565352 20447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:14.565403 20447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:14.565415 20447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:14.565428 20447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:14.565438 20447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:14.567406 20447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:36037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:35993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:35993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:14.567708 20447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:14.567937 20447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:14.573220 20455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:14.573232 20452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:14.573271 20453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:14.573451 20447 server_base.cc:1034] running on GCE node
I20260619 02:45:14.573812 20447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:14.574137 20447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:14.575312 20447 hybrid_clock.cc:648] HybridClock initialized: now 1781837114575250 us; error 77 us; skew 500 ppm
Jun 19 02:45:14 dist-test-slave-zqpb krb5kdc[20434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837114, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:14.578011 20447 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:14.578523 20447 webserver.cc:460] Webserver started at http://127.19.113.126:37353/ using document root <none> and password file <none>
I20260619 02:45:14.578742 20447 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:14.578781 20447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:14.578912 20447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:14.579978 20447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "dc32e48227c345a89f5fac223f18ded3"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "0bb3e69cd30bbaa1d542d3c1acbd59df"
server_key_iv: "3d1418618aa9be683db2206b8fcff44a"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.580303 20447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "dc32e48227c345a89f5fac223f18ded3"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "0bb3e69cd30bbaa1d542d3c1acbd59df"
server_key_iv: "3d1418618aa9be683db2206b8fcff44a"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.581713 20447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:45:14.582600 20462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:14.583583 20447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:14.583673 20447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "dc32e48227c345a89f5fac223f18ded3"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "0bb3e69cd30bbaa1d542d3c1acbd59df"
server_key_iv: "3d1418618aa9be683db2206b8fcff44a"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.583752 20447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:14.596155 20447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:14.598762 20447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:14.598932 20447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:14.603206 20447 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:35993
I20260619 02:45:14.603250 20514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:35993 every 8 connection(s)
I20260619 02:45:14.603608 20447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:14.604562 20515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:14.606860 20515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3: Bootstrap starting.
I20260619 02:45:14.607482 20515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:14.607725 20515 log.cc:824] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:14.608454 20515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3: No bootstrap required, opened a new log
I20260619 02:45:14.610016 20515 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc32e48227c345a89f5fac223f18ded3" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35993 } }
I20260619 02:45:14.610231 20515 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:14.610255 20515 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc32e48227c345a89f5fac223f18ded3, State: Initialized, Role: FOLLOWER
I20260619 02:45:14.610363 20515 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc32e48227c345a89f5fac223f18ded3" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35993 } }
I20260619 02:45:14.610416 20515 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:14.610445 20515 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:14.610482 20515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:14.611042 20515 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc32e48227c345a89f5fac223f18ded3" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35993 } }
I20260619 02:45:14.611146 20515 leader_election.cc:302] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc32e48227c345a89f5fac223f18ded3; no voters:
I20260619 02:45:14.611306 20515 leader_election.cc:288] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:14.611348 20518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:14.611524 20518 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [term 1 LEADER]: Becoming Leader. State: Replica: dc32e48227c345a89f5fac223f18ded3, State: Running, Role: LEADER
I20260619 02:45:14.611555 20515 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:14.611618 20518 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc32e48227c345a89f5fac223f18ded3" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35993 } }
I20260619 02:45:14.612138 20519 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc32e48227c345a89f5fac223f18ded3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc32e48227c345a89f5fac223f18ded3" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35993 } } }
I20260619 02:45:14.612233 20519 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:14.612233 20520 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc32e48227c345a89f5fac223f18ded3. Latest consensus state: current_term: 1 leader_uuid: "dc32e48227c345a89f5fac223f18ded3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc32e48227c345a89f5fac223f18ded3" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35993 } } }
I20260619 02:45:14.612314 20520 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:14.612432 20527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:14.612968 20527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:14.613252 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 20447
I20260619 02:45:14.613344 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:14.613518 19909 external_mini_cluster.cc:1428] Setting key 2199ccb6f921908bff68f9eb869773f5
I20260619 02:45:14.615108 20527 catalog_manager.cc:1347] Generated new cluster ID: d2c2a7ce93ae49a98dea9c1d0451ea77
I20260619 02:45:14.615165 20527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:14 dist-test-slave-zqpb krb5kdc[20434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:14.623474 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:14.614093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.19.113.126:35993)
0619 02:45:14.615020 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:14.615030 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:14.615064 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:45:14.615173 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:14.615179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:14.615239 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:14.615376 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:14.616026 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.616918 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.617723 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.618163 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.618257 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:14.620449 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:14.620453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:14.620453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:14.620466 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:14.621987 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:14.622313 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:14.622316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:14.622316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:14.622413 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:14.622592 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:14.622593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:14.622593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:14.622834 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:14.623028 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:14.623145 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:45:14.623287 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":869,"thread_start_us":64,"threads_started":1}
I20260619 02:45:14.627959 20527 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:14.628594 20527 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:14.636511 20527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc32e48227c345a89f5fac223f18ded3: Generated new TSK 0
I20260619 02:45:14.636708 20527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:14.675732 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35993
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:36037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:14.769043 20541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:14.769464 20541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:14.769507 20541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:14.771358 20541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:14.771410 20541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:14.771443 20541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:14.773113 20541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:36037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:14.773366 20541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:14.773612 20541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:14.779127 20547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:14.779160 20549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:14.779127 20546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:14.779422 20541 server_base.cc:1034] running on GCE node
I20260619 02:45:14.779738 20541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:14.779973 20541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:14.781121 20541 hybrid_clock.cc:648] HybridClock initialized: now 1781837114781108 us; error 27 us; skew 500 ppm
Jun 19 02:45:14 dist-test-slave-zqpb krb5kdc[20434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837114, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:14.783776 20541 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:14.784294 20541 webserver.cc:460] Webserver started at http://127.19.113.65:45761/ using document root <none> and password file <none>
I20260619 02:45:14.784507 20541 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:14.784559 20541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:14.784685 20541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:14.785742 20541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "fe460110ee6e47058fb672a57d7d4e19"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "8cde4a97ea27d59fdef1fb2a2e179a42"
server_key_iv: "68d2ce5962773448de28235dd1779988"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.786089 20541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "fe460110ee6e47058fb672a57d7d4e19"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "8cde4a97ea27d59fdef1fb2a2e179a42"
server_key_iv: "68d2ce5962773448de28235dd1779988"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.787482 20541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:45:14.788360 20556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:14.788625 20541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:14.788717 20541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "fe460110ee6e47058fb672a57d7d4e19"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "8cde4a97ea27d59fdef1fb2a2e179a42"
server_key_iv: "68d2ce5962773448de28235dd1779988"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.788781 20541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:14.819483 20541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:14.821789 20541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:14.821935 20541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:14.822175 20541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:14.822474 20541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:14.822506 20541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:14.822537 20541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:14.822556 20541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:14.828239 20541 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:40147
I20260619 02:45:14.828287 20669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:40147 every 8 connection(s)
I20260619 02:45:14.828601 20541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:14.831602 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 20541
I20260619 02:45:14.831686 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:14.831818 19909 external_mini_cluster.cc:1428] Setting key a6f460bdc00dffb5f4dbd100043db068
Jun 19 02:45:14 dist-test-slave-zqpb krb5kdc[20434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837114, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:14.838804 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:14.829426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:39359 (local address 127.19.113.126:35993)
0619 02:45:14.829493 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:14.829495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:14.829618 (+ 123us) server_negotiation.cc:407] Connection header received
0619 02:45:14.830594 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:14.830595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:14.830612 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:14.830668 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:14.831327 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.831913 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.833299 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.833514 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.833578 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:14.836068 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:14.836069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:14.836070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:14.836078 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:14.837445 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:14.837850 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:14.837851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:14.837851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:14.837908 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:14.838133 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:14.838134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:14.838134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:14.838251 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:14.838307 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:14.838606 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 02:45:14.838706 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:45:14.839010 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:14.829323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:35993 (local address 127.19.113.65:39359)
0619 02:45:14.829497 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:14.829539 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:14.830468 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:14.830728 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:14.830731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:14.831040 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:14.831243 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:14.831247 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.832005 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.832006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:14.833188 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:14.833191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.833595 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.833596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:14.833743 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:14.834309 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:14.834318 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:14.835903 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:14.837535 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:14.837543 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:14.837547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:14.837780 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:14.837963 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:14.837964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:14.837964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:14.838078 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:14.838306 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:14.838312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:14.838475 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 02:45:14.838576 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 02:45:14.838746 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260619 02:45:14.839519 20670 heartbeater.cc:347] Connected to a master server at 127.19.113.126:35993
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
I20260619 02:45:14.839617 20670 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:14.839797 20670 heartbeater.cc:511] Master 127.19.113.126:35993 requested a full tablet report, sending...
I20260619 02:45:14.840375 20479 ts_manager.cc:194] Registered new tserver with Master: fe460110ee6e47058fb672a57d7d4e19 (127.19.113.65:40147)
I20260619 02:45:14.841034 20479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:39359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:14.858472 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35993
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:36037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:14.939361 20675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:14.939581 20675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:14.939606 20675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:14.940771 20675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:14.940807 20675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:14.940833 20675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:14.942306 20675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:36037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:14.942497 20675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:14.942713 20675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:14.947608 20680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:14.947630 20681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:14.947664 20683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:14.947979 20675 server_base.cc:1034] running on GCE node
I20260619 02:45:14.948215 20675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:14.948464 20675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:14.949620 20675 hybrid_clock.cc:648] HybridClock initialized: now 1781837114949613 us; error 38 us; skew 500 ppm
Jun 19 02:45:14 dist-test-slave-zqpb krb5kdc[20434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837114, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:14.952337 20675 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:14.952805 20675 webserver.cc:460] Webserver started at http://127.19.113.66:33279/ using document root <none> and password file <none>
I20260619 02:45:14.953006 20675 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:14.953048 20675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:14.953182 20675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:14.954231 20675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "389644e558754dedbb67c5a5b9229184"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "794d97e2128f020b43c92e73d4b70c98"
server_key_iv: "609848a6c2c14e6e93d484328b262d09"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.954537 20675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "389644e558754dedbb67c5a5b9229184"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "794d97e2128f020b43c92e73d4b70c98"
server_key_iv: "609848a6c2c14e6e93d484328b262d09"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.956169 20675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:45:14.956871 20690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:14.957005 20675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:14.957116 20675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "389644e558754dedbb67c5a5b9229184"
format_stamp: "Formatted at 2026-06-19 02:45:14 on dist-test-slave-zqpb"
server_key: "794d97e2128f020b43c92e73d4b70c98"
server_key_iv: "609848a6c2c14e6e93d484328b262d09"
server_key_version: "encryptionkey@0"
I20260619 02:45:14.957180 20675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:14.981882 20675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:14.983958 20675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:14.984068 20675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:14.984268 20675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:14.984515 20675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:14.984542 20675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:14.984563 20675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:14.984578 20675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:14.990051 20675 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:45587
I20260619 02:45:14.990113 20803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:45587 every 8 connection(s)
I20260619 02:45:14.990406 20675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:14.993465 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 20675
I20260619 02:45:14.993557 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:14.993711 19909 external_mini_cluster.cc:1428] Setting key 5367bdc838a5282169e30459fe9d26b2
Jun 19 02:45:14 dist-test-slave-zqpb krb5kdc[20434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837114, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:15.000921 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:14.991113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:37435 (local address 127.19.113.126:35993)
0619 02:45:14.991158 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:14.991160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:14.991376 (+ 216us) server_negotiation.cc:407] Connection header received
0619 02:45:14.992341 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:14.992342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:14.992355 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:14.992405 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:14.993069 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.993724 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.994937 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.995203 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.995273 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:14.997760 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:14.997762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:14.997762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:14.997771 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:14.999442 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:14.999891 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:14.999892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:14.999893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:14.999962 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.000260 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.000261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.000261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.000423 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:15.000496 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.000735 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.000818 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:15.001139 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:14.991100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:35993 (local address 127.19.113.66:37435)
0619 02:45:14.991261 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:14.991297 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:14.992233 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:14.992477 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:14.992482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:14.992789 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:14.992980 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:14.992984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.993831 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.993832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:14.994843 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:14.994847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:14.995279 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:14.995279 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:14.995383 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:14.996052 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:14.996061 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:14.997644 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:14.999567 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:14.999574 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:14.999576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:14.999810 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:15.000074 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:15.000075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:15.000076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:15.000192 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:15.000488 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:15.000491 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:15.000614 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 02:45:15.000696 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:45:15.000829 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260619 02:45:15.001643 20804 heartbeater.cc:347] Connected to a master server at 127.19.113.126:35993
I20260619 02:45:15.001744 20804 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
I20260619 02:45:15.001935 20804 heartbeater.cc:511] Master 127.19.113.126:35993 requested a full tablet report, sending...
I20260619 02:45:15.002332 20479 ts_manager.cc:194] Registered new tserver with Master: 389644e558754dedbb67c5a5b9229184 (127.19.113.66:45587)
I20260619 02:45:15.002725 20479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:37435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:15.019836 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35993
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:36037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:15.099927 20809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:15.100144 20809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:15.100168 20809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:15.101313 20809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:15.101351 20809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:15.101377 20809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:15.102953 20809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:36037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:35993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:15.103163 20809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:15.103348 20809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:15.108546 20817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:15.108618 20809 server_base.cc:1034] running on GCE node
W20260619 02:45:15.108534 20814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:15.108538 20815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:15.108912 20809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:15.109217 20809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:15.110363 20809 hybrid_clock.cc:648] HybridClock initialized: now 1781837115110350 us; error 32 us; skew 500 ppm
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837115, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:15.113004 20809 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:15.113463 20809 webserver.cc:460] Webserver started at http://127.19.113.67:36971/ using document root <none> and password file <none>
I20260619 02:45:15.113674 20809 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:15.113723 20809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:15.113912 20809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:15.115101 20809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "a6191da45f864213a3a6e3e84f3aa2a3"
format_stamp: "Formatted at 2026-06-19 02:45:15 on dist-test-slave-zqpb"
server_key: "6ae8d91568b12b4d25c5f69503ab96d5"
server_key_iv: "14898254afc89fccacfab0b1e2f7fcb5"
server_key_version: "encryptionkey@0"
I20260619 02:45:15.115434 20809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "a6191da45f864213a3a6e3e84f3aa2a3"
format_stamp: "Formatted at 2026-06-19 02:45:15 on dist-test-slave-zqpb"
server_key: "6ae8d91568b12b4d25c5f69503ab96d5"
server_key_iv: "14898254afc89fccacfab0b1e2f7fcb5"
server_key_version: "encryptionkey@0"
I20260619 02:45:15.116747 20809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:15.117561 20824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:15.117738 20809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:15.117831 20809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "a6191da45f864213a3a6e3e84f3aa2a3"
format_stamp: "Formatted at 2026-06-19 02:45:15 on dist-test-slave-zqpb"
server_key: "6ae8d91568b12b4d25c5f69503ab96d5"
server_key_iv: "14898254afc89fccacfab0b1e2f7fcb5"
server_key_version: "encryptionkey@0"
I20260619 02:45:15.117897 20809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:15.140198 20809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:15.142166 20809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:15.142302 20809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:15.142520 20809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:15.142829 20809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:15.142863 20809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:15.142910 20809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:15.142942 20809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:15.148787 20809 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:40821
I20260619 02:45:15.148837 20937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:40821 every 8 connection(s)
I20260619 02:45:15.149159 20809 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:15.154708 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 20809
I20260619 02:45:15.154815 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:15.155054 19909 external_mini_cluster.cc:1428] Setting key 40c2f33f429b01670fefdcbf2981bcff
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837115, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:15.159143 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.149815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:47239 (local address 127.19.113.126:35993)
0619 02:45:15.149859 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.149861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.150056 (+ 195us) server_negotiation.cc:407] Connection header received
0619 02:45:15.151074 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.151076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.151089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.151136 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:15.151773 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.152399 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.153775 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.153988 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.154048 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.156640 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:15.156641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:15.156641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:15.156649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:15.157827 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.158206 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.158207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.158207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.158262 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.158481 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.158482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.158482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.158613 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:15.158669 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.158957 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.159048 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:15.159329 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.149783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:35993 (local address 127.19.113.67:47239)
0619 02:45:15.149940 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:15.149977 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:15.150961 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:15.151208 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:15.151212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:15.151481 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:15.151682 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.151687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.152578 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.152579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.153678 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.153682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.154067 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.154068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.154220 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.154990 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:15.154999 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:15.156547 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:15.157903 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:15.157911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:15.157913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:15.158138 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:15.158316 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:15.158316 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:15.158317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:15.158426 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:15.158669 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:15.158676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:15.158818 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:45:15.158935 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 02:45:15.159112 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260619 02:45:15.159891 20938 heartbeater.cc:347] Connected to a master server at 127.19.113.126:35993
I20260619 02:45:15.160003 20938 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:15.160189 20938 heartbeater.cc:511] Master 127.19.113.126:35993 requested a full tablet report, sending...
I20260619 02:45:15.160539 20479 ts_manager.cc:194] Registered new tserver with Master: a6191da45f864213a3a6e3e84f3aa2a3 (127.19.113.67:40821)
I20260619 02:45:15.160944 20479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:47239
I20260619 02:45:15.167034 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:45:15.174536 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.168707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.19.113.126:35993)
0619 02:45:15.168776 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.168778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.168812 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:45:15.168879 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.168880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.168890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.168942 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:15.169532 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.170098 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.170777 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.171029 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.171079 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.172210 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:15.172211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:15.172212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:15.172219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:15.173442 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.173765 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.173767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.173767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.173822 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.173975 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.173976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.173976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.174137 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:15.174224 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.174340 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.174413 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:45:15.176702 20479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:15.178231 20479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:15.184743 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.180592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46238 (local address 127.19.113.67:40821)
0619 02:45:15.180793 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.180798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.181066 (+ 268us) server_negotiation.cc:407] Connection header received
0619 02:45:15.181109 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.181111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.181127 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.181193 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:15.181413 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.182183 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.183716 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.184338 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.184400 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.184463 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.184617 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.184659 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":93,"threads_started":1}
I20260619 02:45:15.184767 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.180492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40821 (local address 127.0.0.1:46238)
0619 02:45:15.180871 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:15.180914 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:15.180981 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:15.181279 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:15.181280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:15.181282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:15.181342 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.181344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.182275 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.182276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.183627 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.183630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.184413 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.184414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.184518 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.184532 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:45:15.184595 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 02:45:15.184654 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":5,"thread_start_us":45,"threads_started":1}
I20260619 02:45:15.185199 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.180781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.19.113.65:40147)
0619 02:45:15.181013 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.181017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.181038 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:45:15.181078 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.181080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.181094 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.181160 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:15.181339 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.182477 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.183750 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.184386 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.184452 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.184511 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.185064 (+ 553us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.185116 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":60,"threads_started":1}
I20260619 02:45:15.185251 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.180507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:45587 (local address 127.0.0.1:33826)
0619 02:45:15.180872 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:15.180913 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:15.180981 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:15.181279 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:15.181280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:15.181284 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:15.181332 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.181334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.182941 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.182943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.183920 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.183922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.184934 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.184937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.185085 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.185089 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:15.185150 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:45:15.185190 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":72,"threads_started":1}
I20260619 02:45:15.185313 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.180665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:40147 (local address 127.0.0.1:41700)
0619 02:45:15.180886 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:15.180918 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:15.180979 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:15.181173 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:15.181176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:15.181178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:15.181261 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.181263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.182587 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.182588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.183684 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.183686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.184547 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.184549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.184609 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.184613 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:15.184661 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:45:15.184706 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":67,"threads_started":1}
I20260619 02:45:15.185559 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.180631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.19.113.66:45587)
0619 02:45:15.180798 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.180801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.181066 (+ 265us) server_negotiation.cc:407] Connection header received
0619 02:45:15.181110 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.181111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.181127 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.181193 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:15.181403 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.182499 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.184006 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.184726 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.184782 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.184834 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.185451 (+ 617us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.185489 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260619 02:45:15.185551 20872 tablet_service.cc:1467] Processing CreateTablet for tablet db34877b98ea4334be65d87ce0857e98 (DEFAULT_TABLE table=test-table [id=1675c124f00f44fca280fa3e3c951e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:15.185961 20872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db34877b98ea4334be65d87ce0857e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:15.186333 20738 tablet_service.cc:1467] Processing CreateTablet for tablet db34877b98ea4334be65d87ce0857e98 (DEFAULT_TABLE table=test-table [id=1675c124f00f44fca280fa3e3c951e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:15.186710 20738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db34877b98ea4334be65d87ce0857e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:15.187453 20604 tablet_service.cc:1467] Processing CreateTablet for tablet db34877b98ea4334be65d87ce0857e98 (DEFAULT_TABLE table=test-table [id=1675c124f00f44fca280fa3e3c951e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:15.187772 20604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db34877b98ea4334be65d87ce0857e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:15.189678 20955 tablet_bootstrap.cc:492] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3: Bootstrap starting.
I20260619 02:45:15.190438 20955 tablet_bootstrap.cc:654] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:15.190500 20957 tablet_bootstrap.cc:492] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19: Bootstrap starting.
I20260619 02:45:15.191170 20955 log.cc:824] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:15.191201 20957 tablet_bootstrap.cc:654] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:15.191502 20956 tablet_bootstrap.cc:492] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184: Bootstrap starting.
I20260619 02:45:15.191675 20957 log.cc:824] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:15.191859 20955 tablet_bootstrap.cc:492] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3: No bootstrap required, opened a new log
I20260619 02:45:15.191929 20955 ts_tablet_manager.cc:1397] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:15.192368 20957 tablet_bootstrap.cc:492] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19: No bootstrap required, opened a new log
I20260619 02:45:15.192427 20957 ts_tablet_manager.cc:1397] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:15.193325 20955 raft_consensus.cc:348] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.193468 20955 raft_consensus.cc:374] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:15.193492 20955 raft_consensus.cc:729] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6191da45f864213a3a6e3e84f3aa2a3, State: Initialized, Role: FOLLOWER
I20260619 02:45:15.193598 20955 consensus_queue.cc:260] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.193864 20955 ts_tablet_manager.cc:1428] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:15.193892 20938 heartbeater.cc:503] Master 127.19.113.126:35993 was elected leader, sending a full tablet report...
I20260619 02:45:15.193982 20957 raft_consensus.cc:348] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.194126 20957 raft_consensus.cc:374] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:15.194159 20957 raft_consensus.cc:729] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe460110ee6e47058fb672a57d7d4e19, State: Initialized, Role: FOLLOWER
I20260619 02:45:15.194279 20957 consensus_queue.cc:260] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.194542 20957 ts_tablet_manager.cc:1428] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:15.194662 20670 heartbeater.cc:503] Master 127.19.113.126:35993 was elected leader, sending a full tablet report...
I20260619 02:45:15.194659 20956 tablet_bootstrap.cc:654] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:15.195315 20956 log.cc:824] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:15.195967 20956 tablet_bootstrap.cc:492] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184: No bootstrap required, opened a new log
I20260619 02:45:15.196038 20956 ts_tablet_manager.cc:1397] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260619 02:45:15.197395 20956 raft_consensus.cc:348] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.197533 20956 raft_consensus.cc:374] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:15.197567 20956 raft_consensus.cc:729] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 389644e558754dedbb67c5a5b9229184, State: Initialized, Role: FOLLOWER
I20260619 02:45:15.197680 20956 consensus_queue.cc:260] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.197911 20956 ts_tablet_manager.cc:1428] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:45:15.197943 20804 heartbeater.cc:503] Master 127.19.113.126:35993 was elected leader, sending a full tablet report...
W20260619 02:45:15.240973 20805 tablet.cc:2369] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:15.302340 20963 raft_consensus.cc:482] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:15.302474 20963 raft_consensus.cc:504] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.302798 20963 leader_election.cc:288] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6191da45f864213a3a6e3e84f3aa2a3 (127.19.113.67:40821), fe460110ee6e47058fb672a57d7d4e19 (127.19.113.65:40147)
I20260619 02:45:15.306762 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.303164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:39459 (local address 127.19.113.65:40147)
0619 02:45:15.303206 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.303208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.303554 (+ 346us) server_negotiation.cc:407] Connection header received
0619 02:45:15.303589 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.303590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.303601 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.303659 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:15.303801 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.304517 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.305825 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.306418 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.306477 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.306484 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.306651 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.306687 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:15.306762 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.303085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:36951 (local address 127.19.113.67:40821)
0619 02:45:15.303153 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.303156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.303223 (+ 67us) server_negotiation.cc:407] Connection header received
0619 02:45:15.303255 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.303257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.303267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.303331 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:15.303606 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.304343 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.305779 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.306363 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.306439 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.306448 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.306650 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.306691 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:45:15.306785 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.303094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40821 (local address 127.19.113.66:36951)
0619 02:45:15.303136 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:15.303148 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:15.303204 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:15.303370 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:15.303372 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:15.303375 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:15.303488 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.303490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.304425 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.304426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.305665 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.305669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.306477 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.306479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.306569 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.306574 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:45:15.306639 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:45:15.306685 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":3}
I20260619 02:45:15.306790 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.303201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:40147 (local address 127.19.113.66:39459)
0619 02:45:15.303406 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:15.303416 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:15.303489 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:15.303674 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:15.303674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:15.303675 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:15.303728 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.303728 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.304590 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.304591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.305742 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:15.305747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.306508 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.306509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:15.306581 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.306584 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:15.306633 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:45:15.306689 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":17,"thread_start_us":117,"threads_started":1}
I20260619 02:45:15.307284 20892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db34877b98ea4334be65d87ce0857e98" candidate_uuid: "389644e558754dedbb67c5a5b9229184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" is_pre_election: true
I20260619 02:45:15.307296 20624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db34877b98ea4334be65d87ce0857e98" candidate_uuid: "389644e558754dedbb67c5a5b9229184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe460110ee6e47058fb672a57d7d4e19" is_pre_election: true
I20260619 02:45:15.307451 20624 raft_consensus.cc:2454] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 389644e558754dedbb67c5a5b9229184 in term 0.
I20260619 02:45:15.307452 20892 raft_consensus.cc:2454] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 389644e558754dedbb67c5a5b9229184 in term 0.
I20260619 02:45:15.307687 20691 leader_election.cc:302] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 389644e558754dedbb67c5a5b9229184, fe460110ee6e47058fb672a57d7d4e19; no voters:
I20260619 02:45:15.307852 20963 raft_consensus.cc:2781] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:15.307921 20963 raft_consensus.cc:482] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:15.307945 20963 raft_consensus.cc:3037] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:15.308691 20963 raft_consensus.cc:504] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.308811 20963 leader_election.cc:288] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [CANDIDATE]: Term 1 election: Requested vote from peers a6191da45f864213a3a6e3e84f3aa2a3 (127.19.113.67:40821), fe460110ee6e47058fb672a57d7d4e19 (127.19.113.65:40147)
I20260619 02:45:15.308985 20892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db34877b98ea4334be65d87ce0857e98" candidate_uuid: "389644e558754dedbb67c5a5b9229184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6191da45f864213a3a6e3e84f3aa2a3"
I20260619 02:45:15.309020 20624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db34877b98ea4334be65d87ce0857e98" candidate_uuid: "389644e558754dedbb67c5a5b9229184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe460110ee6e47058fb672a57d7d4e19"
I20260619 02:45:15.309080 20892 raft_consensus.cc:3037] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:15.309092 20624 raft_consensus.cc:3037] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:15.309938 20892 raft_consensus.cc:2454] T db34877b98ea4334be65d87ce0857e98 P a6191da45f864213a3a6e3e84f3aa2a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 389644e558754dedbb67c5a5b9229184 in term 1.
I20260619 02:45:15.309938 20624 raft_consensus.cc:2454] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 389644e558754dedbb67c5a5b9229184 in term 1.
I20260619 02:45:15.310163 20691 leader_election.cc:302] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 389644e558754dedbb67c5a5b9229184, fe460110ee6e47058fb672a57d7d4e19; no voters:
I20260619 02:45:15.310241 20963 raft_consensus.cc:2781] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:15.310390 20963 raft_consensus.cc:686] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [term 1 LEADER]: Becoming Leader. State: Replica: 389644e558754dedbb67c5a5b9229184, State: Running, Role: LEADER
I20260619 02:45:15.310479 20963 consensus_queue.cc:237] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } }
I20260619 02:45:15.311318 20478 catalog_manager.cc:5495] T db34877b98ea4334be65d87ce0857e98 P 389644e558754dedbb67c5a5b9229184 reported cstate change: term changed from 0 to 1, leader changed from <none> to 389644e558754dedbb67c5a5b9229184 (127.19.113.66). New cstate: current_term: 1 leader_uuid: "389644e558754dedbb67c5a5b9229184" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6191da45f864213a3a6e3e84f3aa2a3" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "389644e558754dedbb67c5a5b9229184" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe460110ee6e47058fb672a57d7d4e19" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40147 } health_report { overall_health: UNKNOWN } } }
W20260619 02:45:15.329285 20671 tablet.cc:2369] T db34877b98ea4334be65d87ce0857e98 P fe460110ee6e47058fb672a57d7d4e19: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:15.336102 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.328489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.19.113.65:40147)
0619 02:45:15.328582 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.328585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.328669 (+ 84us) server_negotiation.cc:407] Connection header received
0619 02:45:15.328698 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.328699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.328710 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.328770 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:15.329429 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.329933 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.330683 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.330900 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.330959 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.333381 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:15.333391 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:15.333391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:15.333399 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:15.335103 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.335421 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.335424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.335424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.335476 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.335629 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.335629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.335629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.335753 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:15.335850 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.335909 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.335971 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:15.358129 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.351366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41712 (local address 127.19.113.65:40147)
0619 02:45:15.351423 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.351425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.351458 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:45:15.351562 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.351563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.351572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.351617 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:15.352325 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.352812 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.353548 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.353733 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.353785 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.356058 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:15.356059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:15.356059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:15.356067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:15.357148 (+ 1081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.357432 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.357433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.357433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.357478 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.357677 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.357677 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.357677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.357771 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:15.357841 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.357996 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.358062 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:15.379606 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:15.372731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.19.113.65:40147)
0619 02:45:15.372816 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:15.372818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:15.372890 (+ 72us) server_negotiation.cc:407] Connection header received
0619 02:45:15.372962 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:15.372963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:15.372974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:15.373025 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:15.373566 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.374033 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.374729 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:15.374963 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:15.375017 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:15.377249 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:15.377250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:15.377250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:15.377258 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:15.378552 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.378860 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.378861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.378862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.378954 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:15.379119 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:15.379120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:15.379120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:15.379218 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:15.379299 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:15.379430 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 02:45:15.379501 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 02:45:15.380049 20584 tablet_service.cc:2908] Rejecting scan request for tablet db34877b98ea4334be65d87ce0857e98: Uninitialized: clean time has not yet been initialized
I20260619 02:45:15.380880 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 20541
I20260619 02:45:15.385807 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 20675
I20260619 02:45:15.389819 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 20809
I20260619 02:45:15.393661 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 20447
2026-06-19T02:45:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2088 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20979](info): set up 2 sockets
Jun 19 02:45:15 dist-test-slave-zqpb krb5kdc[20979](info): commencing operation
krb5kdc: starting...
W20260619 02:45:16.350564 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:16 dist-test-slave-zqpb krb5kdc[20979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:16.462072 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:42235
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:42235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:16.541183 20992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:16.541389 20992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:16.541419 20992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:16.542564 20992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:16.542606 20992 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:45:16.542623 20992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:16.542639 20992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:16.542655 20992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:16.542678 20992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:16.544184 20992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:42235
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:42235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:16.544427 20992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:16.544634 20992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:16.549211 20997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:16.549321 20998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:16.549417 20992 server_base.cc:1034] running on GCE node
W20260619 02:45:16.549309 21000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:16.549655 20992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:16.549966 20992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:16.551112 20992 hybrid_clock.cc:648] HybridClock initialized: now 1781837116551095 us; error 29 us; skew 500 ppm
Jun 19 02:45:16 dist-test-slave-zqpb krb5kdc[20979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837116, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:16.557034 20992 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:16.557484 20992 webserver.cc:460] Webserver started at http://127.19.113.126:43303/ using document root <none> and password file <none>
I20260619 02:45:16.557694 20992 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:16.557741 20992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:16.557863 20992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:16.558933 20992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "f667842b0f074764a3e7e713978e06b4"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "23f678056e52e81b2e0ba12239511e6d"
server_key_iv: "fda54abae7ca90febd37ce4d97576e30"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.559273 20992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "f667842b0f074764a3e7e713978e06b4"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "23f678056e52e81b2e0ba12239511e6d"
server_key_iv: "fda54abae7ca90febd37ce4d97576e30"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.560509 20992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:16.561223 21007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:16.561383 20992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:16.561462 20992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "f667842b0f074764a3e7e713978e06b4"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "23f678056e52e81b2e0ba12239511e6d"
server_key_iv: "fda54abae7ca90febd37ce4d97576e30"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.561527 20992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:16.576505 20992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:16.578511 20992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:16.578655 20992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:16.583112 20992 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:42235
I20260619 02:45:16.583202 21069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:42235 every 8 connection(s)
I20260619 02:45:16.583490 20992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:16.584167 21070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:16.586344 21070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4: Bootstrap starting.
I20260619 02:45:16.586658 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 20992
I20260619 02:45:16.586760 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:16.586951 19909 external_mini_cluster.cc:1428] Setting key 09dc522f4478c23104218b08137b3447
I20260619 02:45:16.587103 21070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:16.587401 21070 log.cc:824] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:16.588184 21070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4: No bootstrap required, opened a new log
I20260619 02:45:16.589895 21070 raft_consensus.cc:348] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f667842b0f074764a3e7e713978e06b4" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42235 } }
I20260619 02:45:16.590051 21070 raft_consensus.cc:374] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:16.590083 21070 raft_consensus.cc:729] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f667842b0f074764a3e7e713978e06b4, State: Initialized, Role: FOLLOWER
I20260619 02:45:16.590216 21070 consensus_queue.cc:260] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f667842b0f074764a3e7e713978e06b4" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42235 } }
I20260619 02:45:16.590288 21070 raft_consensus.cc:388] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:16.590372 21070 raft_consensus.cc:482] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:16.590428 21070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:16.591154 21070 raft_consensus.cc:504] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f667842b0f074764a3e7e713978e06b4" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42235 } }
I20260619 02:45:16.591264 21070 leader_election.cc:302] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f667842b0f074764a3e7e713978e06b4; no voters:
I20260619 02:45:16.591423 21070 leader_election.cc:288] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:16.591475 21075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:16.591632 21075 raft_consensus.cc:686] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [term 1 LEADER]: Becoming Leader. State: Replica: f667842b0f074764a3e7e713978e06b4, State: Running, Role: LEADER
I20260619 02:45:16.591711 21070 sys_catalog.cc:564] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:16.591770 21075 consensus_queue.cc:237] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f667842b0f074764a3e7e713978e06b4" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42235 } }
I20260619 02:45:16.592079 21075 sys_catalog.cc:455] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f667842b0f074764a3e7e713978e06b4. Latest consensus state: current_term: 1 leader_uuid: "f667842b0f074764a3e7e713978e06b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f667842b0f074764a3e7e713978e06b4" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42235 } } }
I20260619 02:45:16.592164 21075 sys_catalog.cc:458] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:16.592310 21077 sys_catalog.cc:455] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f667842b0f074764a3e7e713978e06b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f667842b0f074764a3e7e713978e06b4" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42235 } } }
I20260619 02:45:16.592373 21077 sys_catalog.cc:458] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:16.593127 21082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:16.593629 21082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 02:45:16 dist-test-slave-zqpb krb5kdc[20979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:16.595222 21082 catalog_manager.cc:1347] Generated new cluster ID: 55c7524210614b628ed5dcd9737d273a
I20260619 02:45:16.595280 21082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:16.598593 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.587451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.19.113.126:42235)
0619 02:45:16.587614 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:16.587624 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:16.587659 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:45:16.588197 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:16.588205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:16.588256 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:16.588393 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:16.589131 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.590432 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.591506 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.592163 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.592578 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.594969 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:16.594973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:16.594973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:16.594985 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:16.596896 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.597265 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.597268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.597268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.597378 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.597550 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.597551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.597551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.597856 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:16.598045 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:16.598116 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:45:16.598251 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260619 02:45:16.618350 21082 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:16.619024 21082 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:16.630653 21082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4: Generated new TSK 0
I20260619 02:45:16.630851 21082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:16.657370 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42235
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:16.739667 21096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:16.739871 21096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:16.739894 21096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:16.741057 21096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:16.741096 21096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:16.741115 21096 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:16.741132 21096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:16.742635 21096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:16.742834 21096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:16.743093 21096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:16.747792 21101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:16.747757 21102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:16.747887 21096 server_base.cc:1034] running on GCE node
W20260619 02:45:16.747967 21104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:16.748201 21096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:16.748414 21096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:16.749573 21096 hybrid_clock.cc:648] HybridClock initialized: now 1781837116749560 us; error 33 us; skew 500 ppm
Jun 19 02:45:16 dist-test-slave-zqpb krb5kdc[20979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837116, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:16.752251 21096 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:16.752729 21096 webserver.cc:460] Webserver started at http://127.19.113.65:43063/ using document root <none> and password file <none>
I20260619 02:45:16.752948 21096 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:16.752997 21096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:16.753123 21096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:16.754161 21096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "35f52d60de8d4947b13a67c4d2828492"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "3f7baaea725c0dee2b6370943fd9d45a"
server_key_iv: "2234309d2fe8927468ddb46e6f492799"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.754490 21096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "35f52d60de8d4947b13a67c4d2828492"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "3f7baaea725c0dee2b6370943fd9d45a"
server_key_iv: "2234309d2fe8927468ddb46e6f492799"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.755851 21096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:16.756554 21111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:16.756718 21096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:16.756799 21096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "35f52d60de8d4947b13a67c4d2828492"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "3f7baaea725c0dee2b6370943fd9d45a"
server_key_iv: "2234309d2fe8927468ddb46e6f492799"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.756860 21096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:16.775655 21096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:16.777597 21096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:16.777745 21096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:16.778275 21096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:16.778316 21096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:16.778347 21096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:16.778368 21096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:45:16 dist-test-slave-zqpb krb5kdc[20979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837116, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:16.785329 21096 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:39395
I20260619 02:45:16.785408 21226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:39395 every 8 connection(s)
I20260619 02:45:16.785714 21096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:16.787744 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.778713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:33733 (local address 127.19.113.126:42235)
0619 02:45:16.778783 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:16.778785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:16.778976 (+ 191us) server_negotiation.cc:407] Connection header received
0619 02:45:16.779472 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:16.779473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:16.779485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:16.779550 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:16.780206 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.780709 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.782249 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.782500 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.782557 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.785123 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:16.785125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:16.785125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:16.785134 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:16.786598 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.787002 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.787003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.787003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.787055 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.787241 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.787241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.787241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.787344 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:16.787387 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:16.787610 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 02:45:16.787679 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:45:16.787889 21120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.778673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42235 (local address 127.19.113.65:33733)
0619 02:45:16.778825 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:16.778890 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:16.779379 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:16.779631 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:16.779634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:16.779912 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:16.780110 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:16.780115 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.780787 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.780788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:16.782161 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:16.782164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.782600 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.782601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:16.782758 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.783374 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:16.783384 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:16.785017 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:16.786668 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:16.786673 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:16.786675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:16.786931 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:16.787101 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:16.787101 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:16.787102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:16.787195 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:16.787392 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:16.787396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:16.787515 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 02:45:16.787589 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:45:16.787711 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260619 02:45:16.788352 21227 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42235
I20260619 02:45:16.788425 21227 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:16.788584 21227 heartbeater.cc:511] Master 127.19.113.126:42235 requested a full tablet report, sending...
I20260619 02:45:16.789053 21023 ts_manager.cc:194] Registered new tserver with Master: 35f52d60de8d4947b13a67c4d2828492 (127.19.113.65:39395)
I20260619 02:45:16.789754 21023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:33733
I20260619 02:45:16.792251 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 21096
I20260619 02:45:16.792344 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:16.792512 19909 external_mini_cluster.cc:1428] Setting key 155180c0587627c401495abe15f3fe70
I20260619 02:45:16.794706 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.789381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:36523 (local address 127.19.113.126:42235)
0619 02:45:16.789431 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:16.789433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:16.789440 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:16.789506 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:16.789507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:16.789517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:16.789560 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:16.789846 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.790479 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.791263 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.791500 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.791552 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.792363 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:16.792364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:16.792364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:16.792371 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:16.793755 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.794065 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.794066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.794066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.794122 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.794261 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.794261 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.794262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.794373 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:16.794433 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:16.794578 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 02:45:16.794648 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:45:16.794742 21233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.789262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42235 (local address 127.19.113.65:36523)
0619 02:45:16.789389 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:16.789401 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:16.789454 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:16.789562 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:16.789563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:16.789726 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:16.789785 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:16.789786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.790564 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.790565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:16.791193 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:16.791196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.791560 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.791561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:16.791615 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.791939 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:16.791943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:16.792274 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:16.793822 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:16.793823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:16.793823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:16.793974 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:16.794174 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:16.794175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:16.794175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:16.794220 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:16.794425 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:16.794426 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:16.794481 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 02:45:16.794546 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:45:16.794643 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:16.817245 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42235
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:16.896719 21236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:16.896926 21236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:16.896950 21236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:16.898089 21236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:16.898124 21236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:16.898144 21236 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:16.898160 21236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:16.899588 21236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:16.899784 21236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:16.899981 21236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:16.904898 21242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:16.904889 21244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:16.904884 21241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:16.904973 21236 server_base.cc:1034] running on GCE node
I20260619 02:45:16.905441 21236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:16.905678 21236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:16.906818 21236 hybrid_clock.cc:648] HybridClock initialized: now 1781837116906795 us; error 37 us; skew 500 ppm
Jun 19 02:45:16 dist-test-slave-zqpb krb5kdc[20979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837116, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:16.909436 21236 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:16.909858 21236 webserver.cc:460] Webserver started at http://127.19.113.66:40457/ using document root <none> and password file <none>
I20260619 02:45:16.910107 21236 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:16.910153 21236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:16.910241 21236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:16.911324 21236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "e0e8d2045bee4992adf3177ff8986c2e"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "066cf6ae1899d36420c74d7e3750f781"
server_key_iv: "e8e15c1445c2f46bcf9e64b006ce3b98"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.911644 21236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "e0e8d2045bee4992adf3177ff8986c2e"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "066cf6ae1899d36420c74d7e3750f781"
server_key_iv: "e8e15c1445c2f46bcf9e64b006ce3b98"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.912938 21236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:16.913738 21251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:16.913925 21236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:16.914022 21236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "e0e8d2045bee4992adf3177ff8986c2e"
format_stamp: "Formatted at 2026-06-19 02:45:16 on dist-test-slave-zqpb"
server_key: "066cf6ae1899d36420c74d7e3750f781"
server_key_iv: "e8e15c1445c2f46bcf9e64b006ce3b98"
server_key_version: "encryptionkey@0"
I20260619 02:45:16.914084 21236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:16.933754 21236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:16.935963 21236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:16.936102 21236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:16.936602 21236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:16.936646 21236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:16.936676 21236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:16.936693 21236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:16.943233 21236 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:35079
I20260619 02:45:16.943332 21366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:35079 every 8 connection(s)
Jun 19 02:45:16 dist-test-slave-zqpb krb5kdc[20979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837116, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:16.943616 21236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:16.946946 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.937078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:38689 (local address 127.19.113.126:42235)
0619 02:45:16.937126 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:16.937128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:16.937319 (+ 191us) server_negotiation.cc:407] Connection header received
0619 02:45:16.937825 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:16.937826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:16.937839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:16.937897 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:16.938799 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.939598 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.941084 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.941380 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.941434 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.944065 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:16.944067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:16.944067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:16.944074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:16.945293 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.945827 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.945828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.945828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.945886 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.946271 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.946272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.946273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.946402 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:16.946467 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:16.946755 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 02:45:16.946832 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:45:16.947146 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.937063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42235 (local address 127.19.113.66:38689)
0619 02:45:16.937202 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:16.937237 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:16.937728 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:16.938012 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:16.938019 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:16.938369 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:16.938684 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:16.938690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.939687 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.939688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:16.940984 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:16.940989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.941465 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.941466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:16.941622 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.942243 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:16.942255 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:16.943960 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:16.945475 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:16.945484 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:16.945486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:16.945741 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:16.945977 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:16.945978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:16.945978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:16.946178 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:16.946470 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:16.946475 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:16.946612 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 02:45:16.946698 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:45:16.946832 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260619 02:45:16.947582 21367 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42235
I20260619 02:45:16.947667 21367 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:16.947855 21367 heartbeater.cc:511] Master 127.19.113.126:42235 requested a full tablet report, sending...
I20260619 02:45:16.948244 21024 ts_manager.cc:194] Registered new tserver with Master: e0e8d2045bee4992adf3177ff8986c2e (127.19.113.66:35079)
I20260619 02:45:16.948722 21024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:38689
I20260619 02:45:16.952121 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 21236
I20260619 02:45:16.952221 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:16.952420 19909 external_mini_cluster.cc:1428] Setting key 2c46dc8432b3f94e0aed67541d7addab
I20260619 02:45:16.954226 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.948688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:55045 (local address 127.19.113.126:42235)
0619 02:45:16.948735 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:16.948737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:16.948842 (+ 105us) server_negotiation.cc:407] Connection header received
0619 02:45:16.948906 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:16.948907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:16.948916 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:16.949021 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:16.949324 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.949792 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.950470 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.950670 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.950722 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.951686 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:16.951688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:16.951688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:16.951695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:16.953190 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.953494 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.953495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.953495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.953551 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:16.953711 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:16.953712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:16.953712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:16.953860 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:16.953916 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:16.954055 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 02:45:16.954131 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:45:16.954239 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:16.948624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42235 (local address 127.19.113.66:55045)
0619 02:45:16.948791 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:16.948801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:16.948854 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:16.949040 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:16.949041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:16.949204 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:16.949261 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:16.949262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.949874 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.949875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:16.950395 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:16.950396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:16.950726 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:16.950727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:16.950799 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:16.951245 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:16.951250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:16.951550 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:16.953263 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:16.953264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:16.953265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:16.953432 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:16.953601 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:16.953602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:16.953602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:16.953662 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:16.953916 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:16.953917 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:16.953966 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:45:16.954026 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:45:16.954137 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:16.977276 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42235
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:17.057652 21376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:17.057864 21376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:17.057888 21376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:17.059136 21376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:17.059178 21376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:17.059197 21376 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:17.059214 21376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:17.060707 21376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:17.060918 21376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:17.061129 21376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:17.066463 21382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:17.066457 21381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:17.066686 21376 server_base.cc:1034] running on GCE node
W20260619 02:45:17.066463 21384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:17.067031 21376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:17.067306 21376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:17.068513 21376 hybrid_clock.cc:648] HybridClock initialized: now 1781837117068475 us; error 56 us; skew 500 ppm
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[20979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837117, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:17.071202 21376 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:17.071699 21376 webserver.cc:460] Webserver started at http://127.19.113.67:38945/ using document root <none> and password file <none>
I20260619 02:45:17.071924 21376 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:17.071969 21376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:17.072090 21376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:17.073161 21376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "1a8f85e84e7341729845467c5a76e02e"
format_stamp: "Formatted at 2026-06-19 02:45:17 on dist-test-slave-zqpb"
server_key: "824706a4aecdc57ed81502fb84b98655"
server_key_iv: "386b15b476b6ce8c9f03a4b6142b6d06"
server_key_version: "encryptionkey@0"
I20260619 02:45:17.073484 21376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "1a8f85e84e7341729845467c5a76e02e"
format_stamp: "Formatted at 2026-06-19 02:45:17 on dist-test-slave-zqpb"
server_key: "824706a4aecdc57ed81502fb84b98655"
server_key_iv: "386b15b476b6ce8c9f03a4b6142b6d06"
server_key_version: "encryptionkey@0"
I20260619 02:45:17.074769 21376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:17.075575 21391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:17.075726 21376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:17.075802 21376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "1a8f85e84e7341729845467c5a76e02e"
format_stamp: "Formatted at 2026-06-19 02:45:17 on dist-test-slave-zqpb"
server_key: "824706a4aecdc57ed81502fb84b98655"
server_key_iv: "386b15b476b6ce8c9f03a4b6142b6d06"
server_key_version: "encryptionkey@0"
I20260619 02:45:17.075872 21376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:17.094576 21376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:17.096724 21376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:17.096858 21376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:17.097352 21376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:17.097395 21376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:17.097427 21376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:17.097445 21376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:17.103936 21376 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:43297
I20260619 02:45:17.104004 21506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:43297 every 8 connection(s)
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[20979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837117, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:17.104319 21376 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:17.107415 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.097882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:42687 (local address 127.19.113.126:42235)
0619 02:45:17.097946 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.097948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.098209 (+ 261us) server_negotiation.cc:407] Connection header received
0619 02:45:17.098736 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.098737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.098749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.098809 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:17.099667 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.100413 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.101836 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.102128 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.102200 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.104545 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:17.104546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:17.104547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:17.104554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:17.105997 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.106399 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.106400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.106400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.106468 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.106690 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.106691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.106691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.106822 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:17.106897 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.107205 (+ 308us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.107320 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:45:17.107559 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.097859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42235 (local address 127.19.113.67:42687)
0619 02:45:17.098084 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.098122 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.098629 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.098921 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.098926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.099288 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:17.099561 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.099566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.100508 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.100509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.101747 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.101750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.102209 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.102209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.102346 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.102898 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:17.102908 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:17.104419 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:17.106096 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:17.106103 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:17.106105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:17.106329 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:17.106523 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:17.106524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:17.106524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:17.106634 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:17.106904 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:17.106911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:17.107059 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.107168 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.107339 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260619 02:45:17.108163 21507 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42235
I20260619 02:45:17.108253 21507 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:17.108439 21507 heartbeater.cc:511] Master 127.19.113.126:42235 requested a full tablet report, sending...
I20260619 02:45:17.108911 21024 ts_manager.cc:194] Registered new tserver with Master: 1a8f85e84e7341729845467c5a76e02e (127.19.113.67:43297)
I20260619 02:45:17.109660 21024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:42687
I20260619 02:45:17.112134 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 21376
I20260619 02:45:17.112238 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:17.112416 19909 external_mini_cluster.cc:1428] Setting key a86d2c8e84e7ef54f23f28d1ae93ac7f
I20260619 02:45:17.113536 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:45:17.115558 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.109490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:48201 (local address 127.19.113.126:42235)
0619 02:45:17.109542 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.109544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.109554 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:17.109615 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.109616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.109626 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.109667 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:17.110062 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.110659 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.111455 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.111672 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.111733 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.112996 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:17.112998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:17.112998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:17.113004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:17.114389 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.114725 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.114727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.114728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.114785 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.115052 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.115053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.115053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.115166 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:17.115221 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.115361 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.115443 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:45:17.115559 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.109371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42235 (local address 127.19.113.67:48201)
0619 02:45:17.109501 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.109513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.109568 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.109678 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.109679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.109871 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:17.109945 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.109947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.110772 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.110774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.111366 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.111369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.111772 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.111773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.111843 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.112379 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:17.112384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:17.112788 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:17.114493 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:17.114494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:17.114495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:17.114663 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:17.114839 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:17.114839 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:17.114840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:17.114994 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:17.115220 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:17.115221 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:17.115274 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.115334 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.115446 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[20979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837117, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[20979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837117, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:17.136119 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.129314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.19.113.126:42235)
0619 02:45:17.129381 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.129383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.129393 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:17.129420 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.129421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.129431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.129481 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:17.130076 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.130618 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.131402 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.131598 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.131652 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.133842 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:17.133843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:17.133844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:17.133852 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:17.135124 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.135433 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.135434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.135435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.135485 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.135652 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.135653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.135654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.135761 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:17.135850 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.135948 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.136022 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:45:17.137547 21024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:17.139189 21024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:17.145308 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.141410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39395 (local address 127.0.0.1:39674)
0619 02:45:17.141729 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.141767 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.141836 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.142043 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.142044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.142047 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.142110 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.142112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.143125 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.143126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.144152 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.144154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.145024 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.145025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.145102 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.145115 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.145179 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.145227 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":46,"threads_started":1}
I20260619 02:45:17.145421 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.141537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50856 (local address 127.19.113.66:35079)
0619 02:45:17.141699 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.141704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.141839 (+ 135us) server_negotiation.cc:407] Connection header received
0619 02:45:17.141937 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.141940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.141955 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.142028 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.142173 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.143375 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.144410 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.145082 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.145152 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.145200 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.145287 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.145325 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260619 02:45:17.145571 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.141521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39674 (local address 127.19.113.65:39395)
0619 02:45:17.141708 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.141712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.141912 (+ 200us) server_negotiation.cc:407] Connection header received
0619 02:45:17.141952 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.141955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.141967 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.142029 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.142194 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.143057 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.144244 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.144951 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.145009 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.145069 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.145434 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.145477 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
I20260619 02:45:17.145829 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.141414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:35079 (local address 127.0.0.1:50856)
0619 02:45:17.141749 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.141781 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.141868 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.142037 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.142039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.142041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.142108 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.142110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.143471 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.143472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.144330 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.144333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.145158 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.145159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.145213 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.145217 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.145265 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.145297 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":23,"thread_start_us":79,"threads_started":1}
I20260619 02:45:17.145922 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.141707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41556 (local address 127.19.113.67:43297)
0619 02:45:17.141877 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.141881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.142234 (+ 353us) server_negotiation.cc:407] Connection header received
0619 02:45:17.142275 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.142277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.142292 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.142353 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.142516 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.143573 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.144780 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.145464 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.145528 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.145577 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.145797 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.145836 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260619 02:45:17.146148 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.141606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:43297 (local address 127.0.0.1:41556)
0619 02:45:17.142109 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.142118 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.142162 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.142411 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.142412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.142414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.142449 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.142450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.143649 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.143650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.144714 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.144717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.145608 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.145608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.145668 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.145673 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.145719 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.145757 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":65,"threads_started":1}
I20260619 02:45:17.146260 21161 tablet_service.cc:1467] Processing CreateTablet for tablet 353503015c39456ca8662316404cbc05 (DEFAULT_TABLE table=test-table [id=895daa5dadb14393901937bb6afceeea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:17.146433 21301 tablet_service.cc:1467] Processing CreateTablet for tablet 353503015c39456ca8662316404cbc05 (DEFAULT_TABLE table=test-table [id=895daa5dadb14393901937bb6afceeea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:17.146656 21161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353503015c39456ca8662316404cbc05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.146710 21301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353503015c39456ca8662316404cbc05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.146718 21441 tablet_service.cc:1467] Processing CreateTablet for tablet 353503015c39456ca8662316404cbc05 (DEFAULT_TABLE table=test-table [id=895daa5dadb14393901937bb6afceeea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:17.147066 21441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353503015c39456ca8662316404cbc05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.149665 21530 tablet_bootstrap.cc:492] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: Bootstrap starting.
I20260619 02:45:17.150228 21531 tablet_bootstrap.cc:492] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Bootstrap starting.
I20260619 02:45:17.150547 21530 tablet_bootstrap.cc:654] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.150897 21531 tablet_bootstrap.cc:654] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.150907 21530 log.cc:824] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:17.151093 21532 tablet_bootstrap.cc:492] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Bootstrap starting.
I20260619 02:45:17.151217 21531 log.cc:824] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:17.151666 21532 tablet_bootstrap.cc:654] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.151757 21530 tablet_bootstrap.cc:492] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: No bootstrap required, opened a new log
I20260619 02:45:17.151810 21530 ts_tablet_manager.cc:1397] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:17.151988 21532 log.cc:824] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:17.152041 21531 tablet_bootstrap.cc:492] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: No bootstrap required, opened a new log
I20260619 02:45:17.152212 21531 ts_tablet_manager.cc:1397] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:17.152740 21532 tablet_bootstrap.cc:492] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: No bootstrap required, opened a new log
I20260619 02:45:17.152808 21532 ts_tablet_manager.cc:1397] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:17.153326 21530 raft_consensus.cc:348] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.153457 21530 raft_consensus.cc:374] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.153486 21530 raft_consensus.cc:729] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8f85e84e7341729845467c5a76e02e, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.153601 21530 consensus_queue.cc:260] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.153851 21530 ts_tablet_manager.cc:1428] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:17.153884 21507 heartbeater.cc:503] Master 127.19.113.126:42235 was elected leader, sending a full tablet report...
I20260619 02:45:17.154476 21531 raft_consensus.cc:348] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.154649 21531 raft_consensus.cc:374] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.154695 21531 raft_consensus.cc:729] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f52d60de8d4947b13a67c4d2828492, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.154660 21532 raft_consensus.cc:348] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.154789 21532 raft_consensus.cc:374] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.154825 21532 raft_consensus.cc:729] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e8d2045bee4992adf3177ff8986c2e, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.154803 21531 consensus_queue.cc:260] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.154974 21532 consensus_queue.cc:260] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.155074 21531 ts_tablet_manager.cc:1428] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:45:17.155189 21227 heartbeater.cc:503] Master 127.19.113.126:42235 was elected leader, sending a full tablet report...
I20260619 02:45:17.155223 21532 ts_tablet_manager.cc:1428] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:45:17.155326 21367 heartbeater.cc:503] Master 127.19.113.126:42235 was elected leader, sending a full tablet report...
W20260619 02:45:17.194247 21368 tablet.cc:2369] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:45:17.286626 21228 tablet.cc:2369] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:17.302379 21536 raft_consensus.cc:482] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:17.302521 21536 raft_consensus.cc:504] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.302898 21536 leader_election.cc:288] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f52d60de8d4947b13a67c4d2828492 (127.19.113.65:39395), e0e8d2045bee4992adf3177ff8986c2e (127.19.113.66:35079)
I20260619 02:45:17.306769 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.303121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:41949 (local address 127.19.113.65:39395)
0619 02:45:17.303183 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.303185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.303315 (+ 130us) server_negotiation.cc:407] Connection header received
0619 02:45:17.303357 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.303359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.303368 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.303411 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.303595 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.304384 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.305787 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.306387 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.306455 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.306462 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.306663 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.306700 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:45:17.306770 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.303125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:51513 (local address 127.19.113.66:35079)
0619 02:45:17.303169 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.303171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.303209 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:45:17.303285 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.303286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.303295 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.303350 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.303570 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.304321 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.305787 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.306367 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.306452 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.306462 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.306657 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.306699 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:17.306774 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.303093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:35079 (local address 127.19.113.67:51513)
0619 02:45:17.303166 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.303176 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.303225 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.303366 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.303367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.303369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.303440 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.303470 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.304401 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.304402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.305683 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.305686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.306470 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.306472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.306578 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.306581 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.306635 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.306689 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":45}
I20260619 02:45:17.307058 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.303094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39395 (local address 127.19.113.67:41949)
0619 02:45:17.303248 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.303256 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.303309 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.303420 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.303421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.303422 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.303474 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.303475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.304448 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.304449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.305682 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.305686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.306470 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.306472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.306571 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.306576 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.306635 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.306689 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":33,"thread_start_us":92,"threads_started":1}
I20260619 02:45:17.307199 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353503015c39456ca8662316404cbc05" candidate_uuid: "1a8f85e84e7341729845467c5a76e02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e8d2045bee4992adf3177ff8986c2e" is_pre_election: true
I20260619 02:45:17.307358 21321 raft_consensus.cc:2454] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a8f85e84e7341729845467c5a76e02e in term 0.
I20260619 02:45:17.307391 21181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353503015c39456ca8662316404cbc05" candidate_uuid: "1a8f85e84e7341729845467c5a76e02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f52d60de8d4947b13a67c4d2828492" is_pre_election: true
I20260619 02:45:17.307562 21181 raft_consensus.cc:2454] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a8f85e84e7341729845467c5a76e02e in term 0.
I20260619 02:45:17.307739 21394 leader_election.cc:302] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a8f85e84e7341729845467c5a76e02e, e0e8d2045bee4992adf3177ff8986c2e; no voters:
I20260619 02:45:17.307874 21536 raft_consensus.cc:2781] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:17.307937 21536 raft_consensus.cc:482] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:17.307955 21536 raft_consensus.cc:3037] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.308982 21536 raft_consensus.cc:504] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.309182 21536 leader_election.cc:288] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [CANDIDATE]: Term 1 election: Requested vote from peers 35f52d60de8d4947b13a67c4d2828492 (127.19.113.65:39395), e0e8d2045bee4992adf3177ff8986c2e (127.19.113.66:35079)
I20260619 02:45:17.309278 21181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353503015c39456ca8662316404cbc05" candidate_uuid: "1a8f85e84e7341729845467c5a76e02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f52d60de8d4947b13a67c4d2828492"
I20260619 02:45:17.309286 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353503015c39456ca8662316404cbc05" candidate_uuid: "1a8f85e84e7341729845467c5a76e02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e8d2045bee4992adf3177ff8986c2e"
I20260619 02:45:17.309351 21181 raft_consensus.cc:3037] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.309363 21321 raft_consensus.cc:3037] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.310052 21181 raft_consensus.cc:2454] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a8f85e84e7341729845467c5a76e02e in term 1.
I20260619 02:45:17.310120 21321 raft_consensus.cc:2454] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a8f85e84e7341729845467c5a76e02e in term 1.
I20260619 02:45:17.310220 21395 leader_election.cc:302] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a8f85e84e7341729845467c5a76e02e, 35f52d60de8d4947b13a67c4d2828492; no voters:
I20260619 02:45:17.310310 21536 raft_consensus.cc:2781] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:17.310465 21536 raft_consensus.cc:686] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 1 LEADER]: Becoming Leader. State: Replica: 1a8f85e84e7341729845467c5a76e02e, State: Running, Role: LEADER
I20260619 02:45:17.310588 21536 consensus_queue.cc:237] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } }
I20260619 02:45:17.311424 21023 catalog_manager.cc:5495] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a8f85e84e7341729845467c5a76e02e (127.19.113.67). New cstate: current_term: 1 leader_uuid: "1a8f85e84e7341729845467c5a76e02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } health_report { overall_health: UNKNOWN } } }
I20260619 02:45:17.331530 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.329063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.19.113.67:43297)
0619 02:45:17.329118 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.329120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.329129 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:17.329168 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.329169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.329188 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.329246 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:17.329405 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.329877 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.330699 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.330941 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.330994 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.331202 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:17.331303 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:17.331368 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.331413 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.331435 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:45:17.333330 21181 raft_consensus.cc:1261] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Refusing update from remote peer 1a8f85e84e7341729845467c5a76e02e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:17.333340 21321 raft_consensus.cc:1261] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Refusing update from remote peer 1a8f85e84e7341729845467c5a76e02e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:17.333653 21536 consensus_queue.cc:1035] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:17.333796 21540 consensus_queue.cc:1035] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:17.336663 21545 mvcc.cc:205] Tried to move back new op lower bound from 7298404832594554880 to 7298404832504717312. Current Snapshot: MvccSnapshot[applied={T|T < 7298404832594554880}]
I20260619 02:45:17.339053 21023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:17.339138 21023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:17.339788 21023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4: Sending DeleteTablet for 3 replicas of tablet 353503015c39456ca8662316404cbc05
I20260619 02:45:17.340085 21301 tablet_service.cc:1514] Processing DeleteTablet for tablet 353503015c39456ca8662316404cbc05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:17 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:50856
I20260619 02:45:17.340109 21161 tablet_service.cc:1514] Processing DeleteTablet for tablet 353503015c39456ca8662316404cbc05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:17 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:39674
I20260619 02:45:17.340317 21441 tablet_service.cc:1514] Processing DeleteTablet for tablet 353503015c39456ca8662316404cbc05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:17 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:41556
I20260619 02:45:17.340317 21023 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:17.340456 21023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:17.340509 21555 tablet_replica.cc:330] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: stopping tablet replica
I20260619 02:45:17.340588 21555 raft_consensus.cc:2229] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:17.340687 21555 raft_consensus.cc:2258] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:17.340801 21553 tablet_replica.cc:330] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: stopping tablet replica
I20260619 02:45:17.340893 21553 raft_consensus.cc:2229] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:17.341003 21553 raft_consensus.cc:2258] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:17.341473 21555 ts_tablet_manager.cc:1905] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:17.341602 21161 tablet_service.cc:1467] Processing CreateTablet for tablet eb4a5ac019434ccebc4b8c4908493133 (DEFAULT_TABLE table=test-table [id=47668a5b51fd4242911ffbb8c8d7e3ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:17.341691 21301 tablet_service.cc:1467] Processing CreateTablet for tablet eb4a5ac019434ccebc4b8c4908493133 (DEFAULT_TABLE table=test-table [id=47668a5b51fd4242911ffbb8c8d7e3ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:17.341719 21161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4a5ac019434ccebc4b8c4908493133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.341773 21441 tablet_service.cc:1467] Processing CreateTablet for tablet eb4a5ac019434ccebc4b8c4908493133 (DEFAULT_TABLE table=test-table [id=47668a5b51fd4242911ffbb8c8d7e3ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:17.341804 21301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4a5ac019434ccebc4b8c4908493133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.341854 21441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4a5ac019434ccebc4b8c4908493133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.343055 21531 tablet_bootstrap.cc:492] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: Bootstrap starting.
I20260619 02:45:17.343055 21530 tablet_bootstrap.cc:492] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: Bootstrap starting.
I20260619 02:45:17.343427 21531 tablet_bootstrap.cc:654] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.343427 21530 tablet_bootstrap.cc:654] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.343456 21532 tablet_bootstrap.cc:492] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: Bootstrap starting.
I20260619 02:45:17.343808 21532 tablet_bootstrap.cc:654] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.344182 21553 ts_tablet_manager.cc:1905] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:17.344254 21530 tablet_bootstrap.cc:492] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: No bootstrap required, opened a new log
I20260619 02:45:17.344300 21530 ts_tablet_manager.cc:1397] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:17.344442 21530 raft_consensus.cc:348] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.344537 21530 raft_consensus.cc:374] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.344574 21530 raft_consensus.cc:729] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8f85e84e7341729845467c5a76e02e, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.344614 21530 consensus_queue.cc:260] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.344696 21530 ts_tablet_manager.cc:1428] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:17.344403 21531 tablet_bootstrap.cc:492] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: No bootstrap required, opened a new log
I20260619 02:45:17.344781 21531 ts_tablet_manager.cc:1397] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:17.344939 21554 tablet_replica.cc:330] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: stopping tablet replica
I20260619 02:45:17.344933 21531 raft_consensus.cc:348] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.344992 21531 raft_consensus.cc:374] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.345012 21531 raft_consensus.cc:729] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f52d60de8d4947b13a67c4d2828492, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.345033 21554 raft_consensus.cc:2229] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:17.345049 21531 consensus_queue.cc:260] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.345136 21554 raft_consensus.cc:2258] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:17.345145 21531 ts_tablet_manager.cc:1428] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:17.345312 21532 tablet_bootstrap.cc:492] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: No bootstrap required, opened a new log
I20260619 02:45:17.345352 21532 ts_tablet_manager.cc:1397] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:17.345480 21532 raft_consensus.cc:348] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.345534 21554 ts_tablet_manager.cc:1905] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:17.345541 21532 raft_consensus.cc:374] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.345597 21532 raft_consensus.cc:729] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e8d2045bee4992adf3177ff8986c2e, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.345631 21532 consensus_queue.cc:260] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.345712 21532 ts_tablet_manager.cc:1428] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:17.347178 21554 ts_tablet_manager.cc:1918] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:17.347249 21554 log.cc:1196] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/353503015c39456ca8662316404cbc05
I20260619 02:45:17.347500 21554 ts_tablet_manager.cc:1939] T 353503015c39456ca8662316404cbc05 P e0e8d2045bee4992adf3177ff8986c2e: Deleting consensus metadata
I20260619 02:45:17.347904 21010 catalog_manager.cc:4841] TS e0e8d2045bee4992adf3177ff8986c2e (127.19.113.66:35079): tablet 353503015c39456ca8662316404cbc05 (table test-table [id=895daa5dadb14393901937bb6afceeea]) successfully deleted
I20260619 02:45:17.348040 21555 ts_tablet_manager.cc:1918] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:17.348114 21555 log.cc:1196] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/353503015c39456ca8662316404cbc05
I20260619 02:45:17.348338 21555 ts_tablet_manager.cc:1939] T 353503015c39456ca8662316404cbc05 P 1a8f85e84e7341729845467c5a76e02e: Deleting consensus metadata
I20260619 02:45:17.348732 21011 catalog_manager.cc:4841] TS 1a8f85e84e7341729845467c5a76e02e (127.19.113.67:43297): tablet 353503015c39456ca8662316404cbc05 (table test-table [id=895daa5dadb14393901937bb6afceeea]) successfully deleted
I20260619 02:45:17.349758 21553 ts_tablet_manager.cc:1918] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:17.349833 21553 log.cc:1196] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/353503015c39456ca8662316404cbc05
I20260619 02:45:17.350380 21553 ts_tablet_manager.cc:1939] T 353503015c39456ca8662316404cbc05 P 35f52d60de8d4947b13a67c4d2828492: Deleting consensus metadata
I20260619 02:45:17.350780 21011 catalog_manager.cc:4841] TS 35f52d60de8d4947b13a67c4d2828492 (127.19.113.65:39395): tablet 353503015c39456ca8662316404cbc05 (table test-table [id=895daa5dadb14393901937bb6afceeea]) successfully deleted
W20260619 02:45:17.354992 21508 tablet.cc:2369] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:17.396745 21537 raft_consensus.cc:482] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:17.396844 21537 raft_consensus.cc:504] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.397178 21537 leader_election.cc:288] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0e8d2045bee4992adf3177ff8986c2e (127.19.113.66:35079), 1a8f85e84e7341729845467c5a76e02e (127.19.113.67:43297)
I20260619 02:45:17.400609 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.397351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:43297 (local address 127.19.113.65:39815)
0619 02:45:17.397506 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.397516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.397602 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.397779 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.397780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.397782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.397857 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.397859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.398687 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.398688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.399807 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.399809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.400309 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.400310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.400386 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.400391 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.400471 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.400516 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":57,"threads_started":1}
I20260619 02:45:17.400660 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.397409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:39815 (local address 127.19.113.67:43297)
0619 02:45:17.397457 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.397459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.397575 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:45:17.397687 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.397689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.397702 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.397763 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.397925 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.398606 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.399877 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.400233 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.400294 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.400305 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.400511 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.400555 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:45:17.400699 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.397531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:40713 (local address 127.19.113.66:35079)
0619 02:45:17.397587 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.397589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.397600 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:17.397685 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.397686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.397696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.397723 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.397950 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.398418 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.399799 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.400336 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.400420 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.400429 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.400592 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.400640 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:17.400858 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4a5ac019434ccebc4b8c4908493133" candidate_uuid: "35f52d60de8d4947b13a67c4d2828492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a8f85e84e7341729845467c5a76e02e" is_pre_election: true
I20260619 02:45:17.400979 21461 raft_consensus.cc:2454] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f52d60de8d4947b13a67c4d2828492 in term 0.
I20260619 02:45:17.400987 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.397355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:35079 (local address 127.19.113.65:40713)
0619 02:45:17.397548 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.397557 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.397602 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.397783 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.397783 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.397784 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.397861 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.397862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.398503 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.398504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.399696 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.399699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.400432 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.400433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.400510 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.400513 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.400766 (+ 253us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.400810 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":15,"thread_start_us":71,"threads_started":1}
I20260619 02:45:17.401183 21115 leader_election.cc:302] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a8f85e84e7341729845467c5a76e02e, 35f52d60de8d4947b13a67c4d2828492; no voters:
I20260619 02:45:17.401330 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4a5ac019434ccebc4b8c4908493133" candidate_uuid: "35f52d60de8d4947b13a67c4d2828492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e8d2045bee4992adf3177ff8986c2e" is_pre_election: true
I20260619 02:45:17.401356 21537 raft_consensus.cc:2781] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:17.401404 21537 raft_consensus.cc:482] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:17.401412 21321 raft_consensus.cc:2454] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f52d60de8d4947b13a67c4d2828492 in term 0.
I20260619 02:45:17.401422 21537 raft_consensus.cc:3037] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.402189 21537 raft_consensus.cc:504] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.402313 21537 leader_election.cc:288] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [CANDIDATE]: Term 1 election: Requested vote from peers e0e8d2045bee4992adf3177ff8986c2e (127.19.113.66:35079), 1a8f85e84e7341729845467c5a76e02e (127.19.113.67:43297)
I20260619 02:45:17.402496 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4a5ac019434ccebc4b8c4908493133" candidate_uuid: "35f52d60de8d4947b13a67c4d2828492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a8f85e84e7341729845467c5a76e02e"
I20260619 02:45:17.402491 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4a5ac019434ccebc4b8c4908493133" candidate_uuid: "35f52d60de8d4947b13a67c4d2828492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e8d2045bee4992adf3177ff8986c2e"
I20260619 02:45:17.402562 21461 raft_consensus.cc:3037] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.402575 21321 raft_consensus.cc:3037] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.403187 21461 raft_consensus.cc:2454] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f52d60de8d4947b13a67c4d2828492 in term 1.
I20260619 02:45:17.403342 21115 leader_election.cc:302] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a8f85e84e7341729845467c5a76e02e, 35f52d60de8d4947b13a67c4d2828492; no voters:
I20260619 02:45:17.403381 21321 raft_consensus.cc:2454] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f52d60de8d4947b13a67c4d2828492 in term 1.
I20260619 02:45:17.403431 21537 raft_consensus.cc:2781] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:17.403578 21537 raft_consensus.cc:686] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 1 LEADER]: Becoming Leader. State: Replica: 35f52d60de8d4947b13a67c4d2828492, State: Running, Role: LEADER
I20260619 02:45:17.403646 21537 consensus_queue.cc:237] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.404302 21023 catalog_manager.cc:5495] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35f52d60de8d4947b13a67c4d2828492 (127.19.113.65). New cstate: current_term: 1 leader_uuid: "35f52d60de8d4947b13a67c4d2828492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[20979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837116, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:17.442945 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.435688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.19.113.126:42235)
0619 02:45:17.435740 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.435743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.435811 (+ 68us) server_negotiation.cc:407] Connection header received
0619 02:45:17.435854 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.435854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.435865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.435940 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:17.436444 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.437184 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.437899 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.438161 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.438229 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.440444 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:17.440445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:17.440445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:17.440451 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:17.441707 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.442127 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.442127 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.442128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.442172 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.442369 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.442369 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.442370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.442465 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:17.442525 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.442686 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.442762 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:45:17.443065 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.435601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42235 (local address 127.0.0.1:47538)
0619 02:45:17.435742 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.435751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.435808 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.435975 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.435976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.436300 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:17.436353 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.436355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.437274 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.437275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.437816 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.437818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.438247 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.438248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.438298 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.438861 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:17.438897 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:17.440356 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:17.441788 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:17.441796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:17.441798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:17.442060 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:17.442238 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:17.442239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:17.442239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:17.442312 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:17.442519 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:17.442526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:17.442587 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.442659 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.442834 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":52,"thread_start_us":69,"threads_started":1}
I20260619 02:45:17.444934 21023 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:47538:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:45:17.445147 21023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:17.446415 21161 tablet_service.cc:1467] Processing CreateTablet for tablet 36660c85dc044d2ba3fd756e3c2a9c05 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d4317cbc95a4a49adc40442c2c837a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:17.446430 21301 tablet_service.cc:1467] Processing CreateTablet for tablet 36660c85dc044d2ba3fd756e3c2a9c05 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d4317cbc95a4a49adc40442c2c837a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:17.446456 21441 tablet_service.cc:1467] Processing CreateTablet for tablet 36660c85dc044d2ba3fd756e3c2a9c05 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d4317cbc95a4a49adc40442c2c837a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:17.446578 21441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36660c85dc044d2ba3fd756e3c2a9c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.446578 21161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36660c85dc044d2ba3fd756e3c2a9c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.446585 21301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36660c85dc044d2ba3fd756e3c2a9c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:17.448086 21532 tablet_bootstrap.cc:492] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: Bootstrap starting.
I20260619 02:45:17.448087 21530 tablet_bootstrap.cc:492] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: Bootstrap starting.
I20260619 02:45:17.448105 21531 tablet_bootstrap.cc:492] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: Bootstrap starting.
I20260619 02:45:17.448479 21532 tablet_bootstrap.cc:654] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.448494 21531 tablet_bootstrap.cc:654] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.448494 21530 tablet_bootstrap.cc:654] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:17.449134 21531 tablet_bootstrap.cc:492] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: No bootstrap required, opened a new log
I20260619 02:45:17.449188 21531 ts_tablet_manager.cc:1397] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:17.449331 21531 raft_consensus.cc:348] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.449399 21531 raft_consensus.cc:374] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.449416 21531 raft_consensus.cc:729] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f52d60de8d4947b13a67c4d2828492, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.449453 21531 consensus_queue.cc:260] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.449483 21530 tablet_bootstrap.cc:492] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: No bootstrap required, opened a new log
I20260619 02:45:17.449527 21531 ts_tablet_manager.cc:1428] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:17.449536 21530 ts_tablet_manager.cc:1397] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:17.449618 21532 tablet_bootstrap.cc:492] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: No bootstrap required, opened a new log
I20260619 02:45:17.449653 21532 ts_tablet_manager.cc:1397] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:17.449779 21532 raft_consensus.cc:348] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.449841 21532 raft_consensus.cc:374] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.449854 21532 raft_consensus.cc:729] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e8d2045bee4992adf3177ff8986c2e, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.449891 21532 consensus_queue.cc:260] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.449996 21532 ts_tablet_manager.cc:1428] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:17.450151 21537 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.449681 21530 raft_consensus.cc:348] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.450232 21537 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:17.450235 21530 raft_consensus.cc:374] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:17.450261 21530 raft_consensus.cc:729] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8f85e84e7341729845467c5a76e02e, State: Initialized, Role: FOLLOWER
I20260619 02:45:17.450302 21530 consensus_queue.cc:260] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.450403 21530 ts_tablet_manager.cc:1428] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:17.450384 21538 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.450460 21538 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:17.450646 21548 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.450718 21548 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:17.548748 21538 raft_consensus.cc:482] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:17.548844 21538 raft_consensus.cc:504] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.549127 21538 leader_election.cc:288] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f52d60de8d4947b13a67c4d2828492 (127.19.113.65:39395), 1a8f85e84e7341729845467c5a76e02e (127.19.113.67:43297)
I20260619 02:45:17.551981 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.549351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:34753 (local address 127.19.113.65:39395)
0619 02:45:17.549401 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.549403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.549488 (+ 85us) server_negotiation.cc:407] Connection header received
0619 02:45:17.549596 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.549597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.549607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.549664 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.549816 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.550339 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.551324 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.551678 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.551746 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.551755 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.551881 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.551918 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:45:17.551982 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.549585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:36719 (local address 127.19.113.67:43297)
0619 02:45:17.549632 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.549633 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.549641 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:17.549688 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.549689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.549697 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.549750 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:17.549873 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.550343 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.551337 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.551686 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.551747 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.551755 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.551879 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.551917 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:45:17.552023 21573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.549452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:43297 (local address 127.19.113.66:36719)
0619 02:45:17.549552 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.549562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.549633 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.549755 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.549755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.549757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.549813 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.549814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.550419 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.550420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.551248 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.551250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.551755 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.551756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.551815 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.551818 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.551862 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.551904 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260619 02:45:17.552026 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.549291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39395 (local address 127.19.113.66:34753)
0619 02:45:17.549404 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.549415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.549494 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.549670 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.549671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.549673 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:17.549753 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.549755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.550422 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.550423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.551245 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.551247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.551751 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.551751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.551818 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.551821 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.551869 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.551911 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":29,"thread_start_us":57,"threads_started":1}
I20260619 02:45:17.552310 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36660c85dc044d2ba3fd756e3c2a9c05" candidate_uuid: "e0e8d2045bee4992adf3177ff8986c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a8f85e84e7341729845467c5a76e02e" is_pre_election: true
I20260619 02:45:17.552389 21461 raft_consensus.cc:2454] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e8d2045bee4992adf3177ff8986c2e in term 0.
I20260619 02:45:17.552386 21181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36660c85dc044d2ba3fd756e3c2a9c05" candidate_uuid: "e0e8d2045bee4992adf3177ff8986c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f52d60de8d4947b13a67c4d2828492" is_pre_election: true
I20260619 02:45:17.552476 21181 raft_consensus.cc:2454] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e8d2045bee4992adf3177ff8986c2e in term 0.
I20260619 02:45:17.552561 21255 leader_election.cc:302] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a8f85e84e7341729845467c5a76e02e, e0e8d2045bee4992adf3177ff8986c2e; no voters:
I20260619 02:45:17.552697 21538 raft_consensus.cc:2781] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:17.552737 21538 raft_consensus.cc:482] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:17.552757 21538 raft_consensus.cc:3037] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.553424 21538 raft_consensus.cc:504] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.553546 21538 leader_election.cc:288] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [CANDIDATE]: Term 1 election: Requested vote from peers 35f52d60de8d4947b13a67c4d2828492 (127.19.113.65:39395), 1a8f85e84e7341729845467c5a76e02e (127.19.113.67:43297)
I20260619 02:45:17.553695 21181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36660c85dc044d2ba3fd756e3c2a9c05" candidate_uuid: "e0e8d2045bee4992adf3177ff8986c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f52d60de8d4947b13a67c4d2828492"
I20260619 02:45:17.553756 21181 raft_consensus.cc:3037] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.553732 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36660c85dc044d2ba3fd756e3c2a9c05" candidate_uuid: "e0e8d2045bee4992adf3177ff8986c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a8f85e84e7341729845467c5a76e02e"
I20260619 02:45:17.553808 21461 raft_consensus.cc:3037] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:17.554333 21181 raft_consensus.cc:2454] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e8d2045bee4992adf3177ff8986c2e in term 1.
I20260619 02:45:17.554497 21255 leader_election.cc:302] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f52d60de8d4947b13a67c4d2828492, e0e8d2045bee4992adf3177ff8986c2e; no voters:
I20260619 02:45:17.554495 21461 raft_consensus.cc:2454] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e8d2045bee4992adf3177ff8986c2e in term 1.
I20260619 02:45:17.554627 21538 raft_consensus.cc:2781] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:17.554726 21538 raft_consensus.cc:686] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [term 1 LEADER]: Becoming Leader. State: Replica: e0e8d2045bee4992adf3177ff8986c2e, State: Running, Role: LEADER
I20260619 02:45:17.554809 21538 consensus_queue.cc:237] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } }
I20260619 02:45:17.555142 21574 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: TxnStatusTablet state changed. Reason: New leader e0e8d2045bee4992adf3177ff8986c2e. Latest consensus state: current_term: 1 leader_uuid: "e0e8d2045bee4992adf3177ff8986c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.555218 21574 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:17.555380 21576 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:45:17.555516 21576 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:45:17.555511 21024 catalog_manager.cc:5495] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e reported cstate change: term changed from 0 to 1, leader changed from <none> to e0e8d2045bee4992adf3177ff8986c2e (127.19.113.66). New cstate: current_term: 1 leader_uuid: "e0e8d2045bee4992adf3177ff8986c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260619 02:45:17.569377 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.567107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.19.113.66:35079)
0619 02:45:17.567153 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.567155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.567196 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:45:17.567278 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.567279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.567296 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.567348 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:17.567492 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.567936 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.568649 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.568829 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.568879 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.569059 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:17.569145 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:17.569204 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.569257 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.569277 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:45:17.569362 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.567069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:35079 (local address 127.0.0.1:50860)
0619 02:45:17.567152 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.567162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.567213 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.567360 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.567360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.567361 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:17.567411 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.567414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.568011 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.568012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.568574 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.568576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.568886 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.568887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.568955 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.568976 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:17.569197 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:17.569205 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.569244 (+ 39us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.569292 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 02:45:17.570775 21461 raft_consensus.cc:1261] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Refusing update from remote peer e0e8d2045bee4992adf3177ff8986c2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:17.571017 21181 raft_consensus.cc:1261] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Refusing update from remote peer e0e8d2045bee4992adf3177ff8986c2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:17.571071 21574 consensus_queue.cc:1035] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:17.571219 21538 consensus_queue.cc:1035] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:17.571722 21548 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: TxnStatusTablet state changed. Reason: New leader e0e8d2045bee4992adf3177ff8986c2e. Latest consensus state: current_term: 1 leader_uuid: "e0e8d2045bee4992adf3177ff8986c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.571808 21548 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:17.572080 21537 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: TxnStatusTablet state changed. Reason: New leader e0e8d2045bee4992adf3177ff8986c2e. Latest consensus state: current_term: 1 leader_uuid: "e0e8d2045bee4992adf3177ff8986c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.572185 21537 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:17.572664 21538 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0e8d2045bee4992adf3177ff8986c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.572747 21538 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:17.572974 21543 mvcc.cc:205] Tried to move back new op lower bound from 7298404833567137792 to 7298404833505218560. Current Snapshot: MvccSnapshot[applied={T|T < 7298404833567137792}]
I20260619 02:45:17.573096 21548 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0e8d2045bee4992adf3177ff8986c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.573160 21548 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P 1a8f85e84e7341729845467c5a76e02e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:17.573583 21574 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0e8d2045bee4992adf3177ff8986c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.573658 21574 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P e0e8d2045bee4992adf3177ff8986c2e: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:17.574198 21537 tablet_replica.cc:439] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0e8d2045bee4992adf3177ff8986c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f52d60de8d4947b13a67c4d2828492" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39395 } } peers { permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 } } peers { permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 } } }
I20260619 02:45:17.574281 21537 tablet_replica.cc:442] T 36660c85dc044d2ba3fd756e3c2a9c05 P 35f52d60de8d4947b13a67c4d2828492: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:17.577226 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.574587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39676 (local address 127.19.113.65:39395)
0619 02:45:17.574621 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.574623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.574649 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:45:17.574750 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.574751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.574771 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.574830 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:17.574972 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.575725 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.576443 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.576739 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.576803 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.576889 (+ 86us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:17.577017 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:17.577079 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.577113 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.577134 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:45:17.582182 21583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.579569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:35079 (local address 127.19.113.65:56695)
0619 02:45:17.579769 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.579778 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.579828 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.579941 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.579941 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.579943 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:17.580000 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.580001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.580839 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.580840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.581423 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.581425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.581851 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.581852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.581896 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.581909 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:17.582058 (+ 149us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:17.582059 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.582086 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.582119 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260619 02:45:17.582175 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.579638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:56695 (local address 127.19.113.66:35079)
0619 02:45:17.579692 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.579694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.579839 (+ 145us) server_negotiation.cc:407] Connection header received
0619 02:45:17.579865 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.579866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.579876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.579936 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:17.580066 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.580769 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.581493 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.581790 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.581852 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.581971 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:17.582007 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:17.582048 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.582091 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.582107 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:45:17.586771 21461 raft_consensus.cc:1261] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Refusing update from remote peer 35f52d60de8d4947b13a67c4d2828492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:17.586777 21321 raft_consensus.cc:1261] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Refusing update from remote peer 35f52d60de8d4947b13a67c4d2828492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:17.587069 21537 consensus_queue.cc:1035] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a8f85e84e7341729845467c5a76e02e" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:17.587217 21537 consensus_queue.cc:1035] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0e8d2045bee4992adf3177ff8986c2e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 35079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:17.595928 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.593311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:54897 (local address 127.19.113.65:39395)
0619 02:45:17.593352 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.593354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.593504 (+ 150us) server_negotiation.cc:407] Connection header received
0619 02:45:17.593563 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.593564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.593572 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.593613 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:17.593730 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.594189 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.595352 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.595548 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.595596 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.595759 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:17.595782 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:17.595811 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.595860 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.595874 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:17.596010 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.593267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39395 (local address 127.19.113.66:54897)
0619 02:45:17.593438 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:17.593451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:17.593507 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:17.593617 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:17.593618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:17.593619 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:17.593679 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.593680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.594252 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.594253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.595285 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:17.595288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.595607 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.595608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:17.595673 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.595689 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:17.595813 (+ 124us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:17.595815 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:45:17.595859 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:45:17.595901 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260619 02:45:17.605580 21022 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:17.605696 21022 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:17.606382 21022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f667842b0f074764a3e7e713978e06b4: Sending DeleteTablet for 3 replicas of tablet eb4a5ac019434ccebc4b8c4908493133
I20260619 02:45:17.606616 21161 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4a5ac019434ccebc4b8c4908493133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:17 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:39674
I20260619 02:45:17.606730 21553 tablet_replica.cc:330] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: stopping tablet replica
I20260619 02:45:17.606779 21553 raft_consensus.cc:2229] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:17.606742 21441 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4a5ac019434ccebc4b8c4908493133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:17 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:41556
I20260619 02:45:17.606843 21553 raft_consensus.cc:2258] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:17.606837 21301 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4a5ac019434ccebc4b8c4908493133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:17 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:50856
I20260619 02:45:17.606962 21555 tablet_replica.cc:330] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: stopping tablet replica
I20260619 02:45:17.607025 21555 raft_consensus.cc:2229] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:17.607057 21555 raft_consensus.cc:2258] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:17.607127 21554 tablet_replica.cc:330] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: stopping tablet replica
I20260619 02:45:17.607151 21553 ts_tablet_manager.cc:1905] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:17.607177 21554 raft_consensus.cc:2229] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:17.607209 21554 raft_consensus.cc:2258] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:17.607360 21555 ts_tablet_manager.cc:1905] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:17.608454 21554 ts_tablet_manager.cc:1905] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:17.608925 21553 ts_tablet_manager.cc:1918] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:17.608975 21553 log.cc:1196] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/eb4a5ac019434ccebc4b8c4908493133
I20260619 02:45:17.608991 21555 ts_tablet_manager.cc:1918] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:17.609033 21555 log.cc:1196] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/eb4a5ac019434ccebc4b8c4908493133
I20260619 02:45:17.609181 21553 ts_tablet_manager.cc:1939] T eb4a5ac019434ccebc4b8c4908493133 P 35f52d60de8d4947b13a67c4d2828492: Deleting consensus metadata
I20260619 02:45:17.609220 21555 ts_tablet_manager.cc:1939] T eb4a5ac019434ccebc4b8c4908493133 P 1a8f85e84e7341729845467c5a76e02e: Deleting consensus metadata
I20260619 02:45:17.609755 21011 catalog_manager.cc:4841] TS 1a8f85e84e7341729845467c5a76e02e (127.19.113.67:43297): tablet eb4a5ac019434ccebc4b8c4908493133 (table test-table [id=47668a5b51fd4242911ffbb8c8d7e3ab]) successfully deleted
I20260619 02:45:17.609824 21011 catalog_manager.cc:4841] TS 35f52d60de8d4947b13a67c4d2828492 (127.19.113.65:39395): tablet eb4a5ac019434ccebc4b8c4908493133 (table test-table [id=47668a5b51fd4242911ffbb8c8d7e3ab]) successfully deleted
I20260619 02:45:17.610037 21554 ts_tablet_manager.cc:1918] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:17.610085 21554 log.cc:1196] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/eb4a5ac019434ccebc4b8c4908493133
I20260619 02:45:17.610271 21554 ts_tablet_manager.cc:1939] T eb4a5ac019434ccebc4b8c4908493133 P e0e8d2045bee4992adf3177ff8986c2e: Deleting consensus metadata
I20260619 02:45:17.610617 21010 catalog_manager.cc:4841] TS e0e8d2045bee4992adf3177ff8986c2e (127.19.113.66:35079): tablet eb4a5ac019434ccebc4b8c4908493133 (table test-table [id=47668a5b51fd4242911ffbb8c8d7e3ab]) successfully deleted
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[20979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837117, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:17.619724 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.609543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.19.113.65:39395)
0619 02:45:17.609582 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.609585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.609826 (+ 241us) server_negotiation.cc:407] Connection header received
0619 02:45:17.609933 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.609934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.609944 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.609974 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:17.610753 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.611512 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.612581 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.612897 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.612980 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.616318 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:17.616325 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:17.616325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:17.616334 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:17.618333 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.618748 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.618752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.618752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.618819 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.619115 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.619116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.619116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.619303 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:17.619440 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.619514 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.619598 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 02:45:17.620083 21204 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39680
I20260619 02:45:17.628249 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:17.621002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.19.113.126:42235)
0619 02:45:17.621049 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:17.621051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:17.621246 (+ 195us) server_negotiation.cc:407] Connection header received
0619 02:45:17.621280 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:17.621281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:17.621292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:17.621355 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:17.621946 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.622743 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.623722 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:17.624047 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:17.624116 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:17.625443 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:17.625445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:17.625445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:17.625453 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:17.627044 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.627407 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.627408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.627409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.627475 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:17.627661 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:17.627662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:17.627662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:17.627867 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:17.627966 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:17.628067 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:45:17.628150 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:45:17.628829 21022 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47554
I20260619 02:45:17.629556 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 21096
I20260619 02:45:17.634827 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 21236
I20260619 02:45:17.639338 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 21376
I20260619 02:45:17.643885 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 20992
2026-06-19T02:45:17Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2250 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[21600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[21600](info): set up 2 sockets
Jun 19 02:45:17 dist-test-slave-zqpb krb5kdc[21600](info): commencing operation
krb5kdc: starting...
W20260619 02:45:18.598523 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:18 dist-test-slave-zqpb krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:18.717432 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36787
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:36787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:18.796727 21613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:18.796933 21613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:18.796964 21613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:18.798126 21613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:18.798167 21613 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:45:18.798179 21613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:18.798192 21613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:18.798202 21613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:18.798213 21613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:18.799721 21613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:36787
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:18.799958 21613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:18.800184 21613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:18.804881 21618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:18.804931 21613 server_base.cc:1034] running on GCE node
W20260619 02:45:18.804881 21619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:18.804883 21621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:18.805284 21613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:18.805562 21613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:18.806707 21613 hybrid_clock.cc:648] HybridClock initialized: now 1781837118806694 us; error 28 us; skew 500 ppm
Jun 19 02:45:18 dist-test-slave-zqpb krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837118, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:18.809322 21613 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:18.809867 21613 webserver.cc:460] Webserver started at http://127.19.113.126:45509/ using document root <none> and password file <none>
I20260619 02:45:18.810189 21613 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:18.810276 21613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:18.810416 21613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:18.811861 21613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "de59d6dc757c45d4b7a7853c7c0b2c31"
format_stamp: "Formatted at 2026-06-19 02:45:18 on dist-test-slave-zqpb"
server_key: "75f5c09040909d76f15777d1c068c64d"
server_key_iv: "eb24c70e366834a3f77fe9228e02a918"
server_key_version: "encryptionkey@0"
I20260619 02:45:18.812243 21613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "de59d6dc757c45d4b7a7853c7c0b2c31"
format_stamp: "Formatted at 2026-06-19 02:45:18 on dist-test-slave-zqpb"
server_key: "75f5c09040909d76f15777d1c068c64d"
server_key_iv: "eb24c70e366834a3f77fe9228e02a918"
server_key_version: "encryptionkey@0"
I20260619 02:45:18.813411 21613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:45:18.814050 21628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:18.814177 21613 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:18.814262 21613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "de59d6dc757c45d4b7a7853c7c0b2c31"
format_stamp: "Formatted at 2026-06-19 02:45:18 on dist-test-slave-zqpb"
server_key: "75f5c09040909d76f15777d1c068c64d"
server_key_iv: "eb24c70e366834a3f77fe9228e02a918"
server_key_version: "encryptionkey@0"
I20260619 02:45:18.814329 21613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:18.828974 21613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:18.831290 21613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:18.831418 21613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:18.835875 21613 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:36787
I20260619 02:45:18.835942 21690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:36787 every 8 connection(s)
I20260619 02:45:18.836251 21613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:18.836871 21691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:18.839141 21691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: Bootstrap starting.
I20260619 02:45:18.839689 21691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:18.839943 21691 log.cc:824] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:18.840678 21691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: No bootstrap required, opened a new log
I20260619 02:45:18.842101 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 21613
I20260619 02:45:18.842181 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:18.842329 19909 external_mini_cluster.cc:1428] Setting key 5fdfeaba6abab75cdb7d5dfbea42ec67
I20260619 02:45:18.842862 21691 raft_consensus.cc:348] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de59d6dc757c45d4b7a7853c7c0b2c31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36787 } }
I20260619 02:45:18.843086 21691 raft_consensus.cc:374] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:18.843125 21691 raft_consensus.cc:729] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de59d6dc757c45d4b7a7853c7c0b2c31, State: Initialized, Role: FOLLOWER
I20260619 02:45:18.843257 21691 consensus_queue.cc:260] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de59d6dc757c45d4b7a7853c7c0b2c31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36787 } }
I20260619 02:45:18.843334 21691 raft_consensus.cc:388] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:18.843364 21691 raft_consensus.cc:482] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:18.843410 21691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:18.843956 21691 raft_consensus.cc:504] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de59d6dc757c45d4b7a7853c7c0b2c31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36787 } }
I20260619 02:45:18.844063 21691 leader_election.cc:302] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de59d6dc757c45d4b7a7853c7c0b2c31; no voters:
I20260619 02:45:18.844225 21691 leader_election.cc:288] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:18.844290 21696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:18.844487 21691 sys_catalog.cc:564] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:18.844488 21696 raft_consensus.cc:686] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [term 1 LEADER]: Becoming Leader. State: Replica: de59d6dc757c45d4b7a7853c7c0b2c31, State: Running, Role: LEADER
I20260619 02:45:18.844704 21696 consensus_queue.cc:237] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de59d6dc757c45d4b7a7853c7c0b2c31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36787 } }
I20260619 02:45:18.845088 21697 sys_catalog.cc:455] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de59d6dc757c45d4b7a7853c7c0b2c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de59d6dc757c45d4b7a7853c7c0b2c31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36787 } } }
I20260619 02:45:18.845170 21697 sys_catalog.cc:458] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:18.845368 21698 sys_catalog.cc:455] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de59d6dc757c45d4b7a7853c7c0b2c31. Latest consensus state: current_term: 1 leader_uuid: "de59d6dc757c45d4b7a7853c7c0b2c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de59d6dc757c45d4b7a7853c7c0b2c31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36787 } } }
I20260619 02:45:18.845757 21698 sys_catalog.cc:458] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:18.845906 21705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:18.846688 21705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:18.848238 21705 catalog_manager.cc:1347] Generated new cluster ID: 413e3ec7887e4bbe957ff40572bf3777
I20260619 02:45:18.848289 21705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:18 dist-test-slave-zqpb krb5kdc[21600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:18.853157 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:18.842890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.19.113.126:36787)
0619 02:45:18.843069 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:18.843077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:18.843103 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:45:18.843235 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:18.843242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:18.843295 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:18.843412 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:18.844026 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:18.845314 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:18.846289 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:18.846857 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:18.847057 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:18.849656 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:18.849661 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:18.849661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:18.849673 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:18.851593 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:18.851972 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:18.851975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:18.851975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:18.852092 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:18.852262 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:18.852263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:18.852263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:18.852570 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:18.852745 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:18.852812 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 02:45:18.852954 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260619 02:45:18.897102 21705 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:18.897636 21705 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:18.905390 21705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: Generated new TSK 0
I20260619 02:45:18.905561 21705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:18.930088 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36787
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:19.010699 21717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:19.010985 21717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:19.011015 21717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:19.012182 21717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:19.012224 21717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:19.012244 21717 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:45:19.012254 21717 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:19.012269 21717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:19.013716 21717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:36787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:19.013943 21717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:19.014168 21717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:19.019176 21722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:19.019189 21723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:19.019318 21725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:19.019519 21717 server_base.cc:1034] running on GCE node
I20260619 02:45:19.019672 21717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:19.019877 21717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:19.021027 21717 hybrid_clock.cc:648] HybridClock initialized: now 1781837119021013 us; error 23 us; skew 500 ppm
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837119, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:19.023526 21717 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:19.023933 21717 webserver.cc:460] Webserver started at http://127.19.113.65:40337/ using document root <none> and password file <none>
I20260619 02:45:19.024178 21717 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:19.024230 21717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:19.024351 21717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:19.025367 21717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "0050f242fb53482db19392387217e533"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "cac14cd84360cbd58560255d18fd9124"
server_key_iv: "33ddd154e3ab54ce86c0c7a7fa343f66"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.025693 21717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "0050f242fb53482db19392387217e533"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "cac14cd84360cbd58560255d18fd9124"
server_key_iv: "33ddd154e3ab54ce86c0c7a7fa343f66"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.026935 21717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:45:19.027691 21732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.027856 21717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:19.027935 21717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "0050f242fb53482db19392387217e533"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "cac14cd84360cbd58560255d18fd9124"
server_key_iv: "33ddd154e3ab54ce86c0c7a7fa343f66"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.027997 21717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:19.037727 21717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:19.039760 21717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:19.039891 21717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:19.040364 21717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:19.040403 21717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.040434 21717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:19.040457 21717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837119, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:19.047217 21717 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:46457
I20260619 02:45:19.047276 21847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:46457 every 8 connection(s)
I20260619 02:45:19.047623 21717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:19.049875 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.040834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:58305 (local address 127.19.113.126:36787)
0619 02:45:19.040882 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.040885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.041136 (+ 251us) server_negotiation.cc:407] Connection header received
0619 02:45:19.041613 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.041615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.041631 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.041699 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:19.042466 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.043250 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.044329 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.044537 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.044594 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.047096 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:19.047097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:19.047098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:19.047106 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:19.048688 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.049092 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.049093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.049093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.049140 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.049318 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.049318 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.049318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.049425 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:19.049474 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.049737 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.049810 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:45:19.050118 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.040834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36787 (local address 127.19.113.65:58305)
0619 02:45:19.041009 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.041044 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.041515 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.041759 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.041763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.042089 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:19.042365 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.042372 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.043350 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.043351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.044217 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.044221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.044627 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.044628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.044755 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.045277 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:19.045286 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:19.046977 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:19.048779 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.048787 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.048788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.049027 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.049178 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.049179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.049179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.049265 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.049503 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:19.049509 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:19.049630 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.049715 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.049841 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260619 02:45:19.050606 21848 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36787
I20260619 02:45:19.050679 21848 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:19.050833 21848 heartbeater.cc:511] Master 127.19.113.126:36787 requested a full tablet report, sending...
I20260619 02:45:19.051397 21644 ts_manager.cc:194] Registered new tserver with Master: 0050f242fb53482db19392387217e533 (127.19.113.65:46457)
I20260619 02:45:19.052275 21644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:58305
I20260619 02:45:19.055246 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 21717
I20260619 02:45:19.055348 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:19.055531 19909 external_mini_cluster.cc:1428] Setting key e0eb66f2694ae1ffaf4a0f7732d7bb0e
I20260619 02:45:19.056887 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.051525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36787 (local address 127.19.113.65:44089)
0619 02:45:19.051648 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.051659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.051719 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.051825 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.051826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.052004 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:19.052076 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.052077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.052731 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.052732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.053287 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.053290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.053669 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.053669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.053734 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.054182 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:19.054187 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:19.054486 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:19.055994 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.055995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.055996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.056183 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.056379 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.056380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.056380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.056431 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.056647 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:19.056648 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:19.056692 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.056740 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.056815 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":25,"thread_start_us":70,"threads_started":1}
I20260619 02:45:19.056926 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.051630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:44089 (local address 127.19.113.126:36787)
0619 02:45:19.051667 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.051669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.051705 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:45:19.051771 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.051772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.051780 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.051821 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:19.052130 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.052651 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.053379 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.053577 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.053639 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.054573 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:19.054574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:19.054574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:19.054580 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:19.055928 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.056251 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.056251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.056252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.056316 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.056478 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.056479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.056479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.056595 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:19.056659 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.056764 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.056843 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:19.077637 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36787
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:19.159972 21857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:19.160194 21857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:19.160228 21857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:19.161422 21857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:19.161468 21857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:19.161495 21857 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:45:19.161509 21857 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:19.161532 21857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:19.163079 21857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:36787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:19.163329 21857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:19.163539 21857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:19.168448 21863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:19.168453 21862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:19.168453 21865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:19.168746 21857 server_base.cc:1034] running on GCE node
I20260619 02:45:19.168939 21857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:19.169258 21857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:19.170435 21857 hybrid_clock.cc:648] HybridClock initialized: now 1781837119170416 us; error 29 us; skew 500 ppm
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837119, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:19.173142 21857 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:19.173628 21857 webserver.cc:460] Webserver started at http://127.19.113.66:37249/ using document root <none> and password file <none>
I20260619 02:45:19.173833 21857 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:19.173880 21857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:19.174019 21857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:19.175179 21857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "28210ebc05944a74a01e05f4f42b24e6"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "417a14262749abbd168ed5b4eddd26cc"
server_key_iv: "c8b19d88fa66a99c357f65a532cdfcc2"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.175539 21857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "28210ebc05944a74a01e05f4f42b24e6"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "417a14262749abbd168ed5b4eddd26cc"
server_key_iv: "c8b19d88fa66a99c357f65a532cdfcc2"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.176883 21857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:19.177804 21872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.178170 21857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:19.178272 21857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "28210ebc05944a74a01e05f4f42b24e6"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "417a14262749abbd168ed5b4eddd26cc"
server_key_iv: "c8b19d88fa66a99c357f65a532cdfcc2"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.178349 21857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:19.191026 21857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:19.193213 21857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:19.193373 21857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:19.194000 21857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:19.194053 21857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.194082 21857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:19.194097 21857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837119, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:19.201341 21857 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:40665
I20260619 02:45:19.201400 21987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:40665 every 8 connection(s)
I20260619 02:45:19.201719 21857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:19.203433 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 21857
I20260619 02:45:19.203532 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:19.203730 19909 external_mini_cluster.cc:1428] Setting key 6b503e0c0d6381973ca4ff9ec7f70ce6
I20260619 02:45:19.204298 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.194417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:50135 (local address 127.19.113.126:36787)
0619 02:45:19.194484 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.194487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.194670 (+ 183us) server_negotiation.cc:407] Connection header received
0619 02:45:19.195232 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.195233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.195248 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.195317 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:19.195988 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.196511 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.197716 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.198003 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.198064 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.201272 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:19.201274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:19.201275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:19.201284 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:19.202823 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.203318 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.203319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.203319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.203374 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.203635 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.203636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.203636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.203756 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:19.203817 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.204124 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.204209 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:45:19.204435 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.194381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36787 (local address 127.19.113.66:50135)
0619 02:45:19.194539 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.194584 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.195134 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.195390 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.195394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.195676 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:19.195892 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.195898 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.196690 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.196691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.197621 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.197624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.198082 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.198083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.198216 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.198787 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:19.198806 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:19.201129 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:19.202948 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.202954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.202956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.203238 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.203435 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.203436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.203437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.203573 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.203818 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:19.203825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:19.203975 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.204093 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.204225 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260619 02:45:19.205010 21988 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36787
I20260619 02:45:19.205101 21988 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:19.205288 21988 heartbeater.cc:511] Master 127.19.113.126:36787 requested a full tablet report, sending...
I20260619 02:45:19.205658 21645 ts_manager.cc:194] Registered new tserver with Master: 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665)
I20260619 02:45:19.206126 21645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:50135
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
I20260619 02:45:19.212296 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.206279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:38657 (local address 127.19.113.126:36787)
0619 02:45:19.206330 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.206332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.206340 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:19.206357 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.206358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.206367 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.206415 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:19.206812 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.207345 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.208357 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.208554 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.208633 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.209645 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:19.209646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:19.209646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:19.209653 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:19.211108 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.211538 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.211539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.211539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.211593 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.211775 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.211775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.211776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.211910 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:19.211958 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.212158 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.212221 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:45:19.212350 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.205984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36787 (local address 127.19.113.66:38657)
0619 02:45:19.206116 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.206127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.206179 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.206433 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.206434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.206649 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:19.206723 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.206726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.207436 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.207437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.208279 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.208282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.208656 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.208657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.208739 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.209144 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:19.209149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:19.209533 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:19.211239 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.211241 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.211241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.211468 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.211649 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.211649 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.211649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.211717 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.211974 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:19.211975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:19.212028 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.212131 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.212237 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:19.230078 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36787
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:19.308717 21997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:19.308928 21997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:19.308964 21997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:19.310141 21997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:19.310181 21997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:19.310210 21997 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:45:19.310226 21997 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:19.310253 21997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:19.311748 21997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:36787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:19.311973 21997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:19.312181 21997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:19.316982 22005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:19.317013 22002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:19.317004 22003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:19.317152 21997 server_base.cc:1034] running on GCE node
I20260619 02:45:19.317340 21997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:19.317603 21997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:19.318773 21997 hybrid_clock.cc:648] HybridClock initialized: now 1781837119318761 us; error 26 us; skew 500 ppm
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837119, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:19.321178 21997 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:19.321568 21997 webserver.cc:460] Webserver started at http://127.19.113.67:41155/ using document root <none> and password file <none>
I20260619 02:45:19.321745 21997 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:19.321784 21997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:19.321872 21997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:19.322846 21997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "1ee2accedc6d49418052fb60d9e88daa"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "472b1f6202953e18de1afdb11ee66bff"
server_key_iv: "5a0dce11bb23f73b05b44bd82ba695fb"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.323203 21997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "1ee2accedc6d49418052fb60d9e88daa"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "472b1f6202953e18de1afdb11ee66bff"
server_key_iv: "5a0dce11bb23f73b05b44bd82ba695fb"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.324433 21997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:19.325114 22012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.325254 21997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:19.325331 21997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "1ee2accedc6d49418052fb60d9e88daa"
format_stamp: "Formatted at 2026-06-19 02:45:19 on dist-test-slave-zqpb"
server_key: "472b1f6202953e18de1afdb11ee66bff"
server_key_iv: "5a0dce11bb23f73b05b44bd82ba695fb"
server_key_version: "encryptionkey@0"
I20260619 02:45:19.325385 21997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:19.349167 21997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:19.351306 21997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:19.351418 21997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:19.351868 21997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:19.351910 21997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.351931 21997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:19.351945 21997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837119, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:19.358431 21997 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:40499
I20260619 02:45:19.358565 22127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:40499 every 8 connection(s)
I20260619 02:45:19.358785 21997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:19.361402 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.352328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:38063 (local address 127.19.113.126:36787)
0619 02:45:19.352370 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.352372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.352614 (+ 242us) server_negotiation.cc:407] Connection header received
0619 02:45:19.353147 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.353148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.353158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.353182 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:19.354112 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.354899 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.355965 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.356265 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.356327 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.358941 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:19.358943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:19.358944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:19.358952 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:19.360286 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.360687 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.360688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.360688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.360736 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.360910 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.360910 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.360911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.361007 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:19.361050 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.361264 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.361326 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:45:19.361580 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.352333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36787 (local address 127.19.113.67:38063)
0619 02:45:19.352475 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.352533 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.353023 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.353336 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.353340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.353698 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:19.354002 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.354007 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.355007 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.355009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.355873 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.355875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.356380 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.356382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.356579 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.357073 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:19.357083 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:19.358818 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:19.360374 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.360379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.360381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.360621 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.360780 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.360780 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.360780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.360869 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.361050 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:19.361052 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:19.361166 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.361246 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.361366 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":84,"threads_started":1}
I20260619 02:45:19.362064 22128 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36787
I20260619 02:45:19.362138 22128 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:19.362303 22128 heartbeater.cc:511] Master 127.19.113.126:36787 requested a full tablet report, sending...
I20260619 02:45:19.362646 21644 ts_manager.cc:194] Registered new tserver with Master: 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499)
I20260619 02:45:19.363153 21644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:38063
I20260619 02:45:19.365031 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 21997
I20260619 02:45:19.365124 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:19.365278 19909 external_mini_cluster.cc:1428] Setting key 6d01354828bf1432f430d79b34cc41d5
I20260619 02:45:19.366118 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:45:19.368080 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.362979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36787 (local address 127.19.113.67:38469)
0619 02:45:19.363156 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.363167 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.363229 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.363385 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.363386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.363527 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:19.363585 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.363586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.364191 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.364192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.364688 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.364690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.365003 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.365004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.365085 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.365458 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:19.365462 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:19.365813 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:19.367185 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.367186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.367187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.367377 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.367530 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.367530 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.367531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.367587 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.367829 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:19.367830 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:19.367872 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.367922 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.368007 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
I20260619 02:45:19.368117 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.363083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:38469 (local address 127.19.113.126:36787)
0619 02:45:19.363125 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.363126 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.363226 (+ 100us) server_negotiation.cc:407] Connection header received
0619 02:45:19.363327 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.363327 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.363337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.363383 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:19.363650 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.364105 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.364753 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.364949 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.365000 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.365883 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:19.365884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:19.365884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:19.365890 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:19.367117 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.367440 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.367441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.367441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.367485 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.367644 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.367644 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.367645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.367772 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:19.367835 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.367953 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.368021 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:19.387663 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.381346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.19.113.126:36787)
0619 02:45:19.381395 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.381397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.381405 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:19.381431 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.381432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.381440 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.381486 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:19.382001 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.382555 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.383331 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.383515 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.383575 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.385624 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:19.385626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:19.385626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:19.385632 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:19.386737 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.387095 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.387096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.387096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.387143 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.387269 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.387269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.387270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.387361 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:19.387429 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.387524 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.387596 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:19.389263 21644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:19.390902 21644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:19.396941 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.392853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40499 (local address 127.0.0.1:37462)
0619 02:45:19.393041 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.393081 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.393146 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.393425 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.393428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.393430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.393484 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.393486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.394768 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.394769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.395938 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.395941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.396642 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.396721 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.396736 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.396799 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.396854 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":10,"thread_start_us":80,"threads_started":1}
I20260619 02:45:19.397058 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.393226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.19.113.66:40665)
0619 02:45:19.393630 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.393634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.393643 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:19.393680 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.393682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.393696 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.393900 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.393956 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.395020 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396004 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.396724 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396796 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.396852 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.396921 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.396957 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":83,"threads_started":1}
I20260619 02:45:19.397034 22142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.393054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:40665 (local address 127.0.0.1:51938)
0619 02:45:19.393231 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.393239 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.393274 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.393789 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.393790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.393793 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.393836 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.393838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.395090 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.395091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.395937 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.395940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.396802 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.396856 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.396859 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.396907 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.396940 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":40,"threads_started":1}
I20260619 02:45:19.397190 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.392855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46457 (local address 127.0.0.1:40640)
0619 02:45:19.393227 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.393237 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.393283 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.393522 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.393523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.393525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.393564 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.393565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.394534 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.394535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.396055 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.396056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.396959 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.397025 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.397029 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.397088 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.397126 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":60,"threads_started":1}
I20260619 02:45:19.397388 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.392975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.19.113.65:46457)
0619 02:45:19.393388 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.393392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.393401 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:19.393435 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.393437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.393448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.393504 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.393650 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.394456 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396126 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.396819 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396877 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.396926 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.397255 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.397295 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":73,"threads_started":1}
I20260619 02:45:19.398016 21922 tablet_service.cc:1467] Processing CreateTablet for tablet e2f3d2e2a764434db39985d68aaf686f (DEFAULT_TABLE table=test-table [id=e601f919272a44be9db9b956bac45b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:19.398036 21782 tablet_service.cc:1467] Processing CreateTablet for tablet e2f3d2e2a764434db39985d68aaf686f (DEFAULT_TABLE table=test-table [id=e601f919272a44be9db9b956bac45b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:19.398397 21922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2f3d2e2a764434db39985d68aaf686f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.398397 21782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2f3d2e2a764434db39985d68aaf686f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.398990 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.392956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.19.113.67:40499)
0619 02:45:19.393208 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.393213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.393224 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:19.393266 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.393268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.393283 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.393362 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.393563 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.394683 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396026 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.396564 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.396630 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.396691 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.397488 (+ 797us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.397540 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":80,"threads_started":1}
I20260619 02:45:19.402103 22148 tablet_bootstrap.cc:492] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: Bootstrap starting.
I20260619 02:45:19.402722 22148 tablet_bootstrap.cc:654] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.402666 22062 tablet_service.cc:1467] Processing CreateTablet for tablet e2f3d2e2a764434db39985d68aaf686f (DEFAULT_TABLE table=test-table [id=e601f919272a44be9db9b956bac45b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:19.402978 22062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2f3d2e2a764434db39985d68aaf686f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.403028 22149 tablet_bootstrap.cc:492] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Bootstrap starting.
I20260619 02:45:19.403090 22148 log.cc:824] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:19.403625 22149 tablet_bootstrap.cc:654] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.403867 22148 tablet_bootstrap.cc:492] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: No bootstrap required, opened a new log
I20260619 02:45:19.403939 22148 ts_tablet_manager.cc:1397] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:45:19.403947 22149 log.cc:824] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:19.404683 22149 tablet_bootstrap.cc:492] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: No bootstrap required, opened a new log
I20260619 02:45:19.404824 22149 ts_tablet_manager.cc:1397] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:19.405654 22148 raft_consensus.cc:348] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.405786 22148 raft_consensus.cc:374] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.405817 22148 raft_consensus.cc:729] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0050f242fb53482db19392387217e533, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.405932 22148 consensus_queue.cc:260] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.406203 22148 ts_tablet_manager.cc:1428] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:45:19.406332 21848 heartbeater.cc:503] Master 127.19.113.126:36787 was elected leader, sending a full tablet report...
I20260619 02:45:19.406653 22152 tablet_bootstrap.cc:492] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: Bootstrap starting.
I20260619 02:45:19.407056 22149 raft_consensus.cc:348] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.407244 22149 raft_consensus.cc:374] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.407289 22149 raft_consensus.cc:729] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28210ebc05944a74a01e05f4f42b24e6, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.407394 22149 consensus_queue.cc:260] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.407653 22149 ts_tablet_manager.cc:1428] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:45:19.407688 21988 heartbeater.cc:503] Master 127.19.113.126:36787 was elected leader, sending a full tablet report...
I20260619 02:45:19.407874 22152 tablet_bootstrap.cc:654] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.408133 22152 log.cc:824] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:19.408762 22152 tablet_bootstrap.cc:492] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: No bootstrap required, opened a new log
I20260619 02:45:19.408821 22152 ts_tablet_manager.cc:1397] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 02:45:19.410141 22152 raft_consensus.cc:348] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.410264 22152 raft_consensus.cc:374] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.410292 22152 raft_consensus.cc:729] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee2accedc6d49418052fb60d9e88daa, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.410394 22152 consensus_queue.cc:260] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.410615 22152 ts_tablet_manager.cc:1428] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:45:19.410655 22128 heartbeater.cc:503] Master 127.19.113.126:36787 was elected leader, sending a full tablet report...
W20260619 02:45:19.452328 21989 tablet.cc:2369] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:19.488876 22154 raft_consensus.cc:482] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:19.489030 22154 raft_consensus.cc:504] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.489393 22154 leader_election.cc:288] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499), 0050f242fb53482db19392387217e533 (127.19.113.65:46457)
I20260619 02:45:19.492173 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.489671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:33273 (local address 127.19.113.65:46457)
0619 02:45:19.489712 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.489714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.489894 (+ 180us) server_negotiation.cc:407] Connection header received
0619 02:45:19.489942 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.489943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.489952 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.490001 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.490158 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.490616 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.491502 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.491854 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.491915 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.491923 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.492055 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.492095 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:19.492185 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.489643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:40863 (local address 127.19.113.67:40499)
0619 02:45:19.489702 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.489704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.489722 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:45:19.489813 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.489814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.489823 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.489871 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.490028 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.490536 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.491501 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.491871 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.491931 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.491939 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.492079 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.492117 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:45:19.492199 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.489590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40499 (local address 127.19.113.66:40863)
0619 02:45:19.489664 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.489677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.489736 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.489882 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.489883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.489884 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.489972 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.489973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.490610 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.490611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.491411 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.491413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.491947 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.491948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.492019 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.492021 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.492064 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.492116 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":2}
I20260619 02:45:19.492213 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.489590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46457 (local address 127.19.113.66:33273)
0619 02:45:19.489819 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.489827 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.489882 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.490010 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.490011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.490012 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.490096 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.490098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.490694 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.490695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.491421 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.491422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.491926 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.491927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.491995 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.491998 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.492044 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.492109 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":71,"thread_start_us":120,"threads_started":1}
I20260619 02:45:19.492544 22082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2f3d2e2a764434db39985d68aaf686f" candidate_uuid: "28210ebc05944a74a01e05f4f42b24e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee2accedc6d49418052fb60d9e88daa" is_pre_election: true
I20260619 02:45:19.492534 21802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2f3d2e2a764434db39985d68aaf686f" candidate_uuid: "28210ebc05944a74a01e05f4f42b24e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0050f242fb53482db19392387217e533" is_pre_election: true
I20260619 02:45:19.492708 21802 raft_consensus.cc:2454] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28210ebc05944a74a01e05f4f42b24e6 in term 0.
I20260619 02:45:19.492708 22082 raft_consensus.cc:2454] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28210ebc05944a74a01e05f4f42b24e6 in term 0.
I20260619 02:45:19.492921 21873 leader_election.cc:302] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee2accedc6d49418052fb60d9e88daa, 28210ebc05944a74a01e05f4f42b24e6; no voters:
I20260619 02:45:19.493084 22154 raft_consensus.cc:2781] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:19.493134 22154 raft_consensus.cc:482] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:19.493149 22154 raft_consensus.cc:3037] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.494145 22154 raft_consensus.cc:504] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.494263 22154 leader_election.cc:288] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [CANDIDATE]: Term 1 election: Requested vote from peers 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499), 0050f242fb53482db19392387217e533 (127.19.113.65:46457)
I20260619 02:45:19.494443 21802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2f3d2e2a764434db39985d68aaf686f" candidate_uuid: "28210ebc05944a74a01e05f4f42b24e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0050f242fb53482db19392387217e533"
I20260619 02:45:19.494517 21802 raft_consensus.cc:3037] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.495227 21802 raft_consensus.cc:2454] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28210ebc05944a74a01e05f4f42b24e6 in term 1.
I20260619 02:45:19.495327 22082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2f3d2e2a764434db39985d68aaf686f" candidate_uuid: "28210ebc05944a74a01e05f4f42b24e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee2accedc6d49418052fb60d9e88daa"
I20260619 02:45:19.495404 21876 leader_election.cc:302] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0050f242fb53482db19392387217e533, 28210ebc05944a74a01e05f4f42b24e6; no voters:
I20260619 02:45:19.495407 22082 raft_consensus.cc:3037] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.495494 22154 raft_consensus.cc:2781] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:19.495669 22154 raft_consensus.cc:686] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 1 LEADER]: Becoming Leader. State: Replica: 28210ebc05944a74a01e05f4f42b24e6, State: Running, Role: LEADER
I20260619 02:45:19.495762 22154 consensus_queue.cc:237] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } }
I20260619 02:45:19.496239 22082 raft_consensus.cc:2454] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28210ebc05944a74a01e05f4f42b24e6 in term 1.
I20260619 02:45:19.496558 21644 catalog_manager.cc:5495] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66). New cstate: current_term: 1 leader_uuid: "28210ebc05944a74a01e05f4f42b24e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:45:19 dist-test-slave-zqpb krb5kdc[21600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837118, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:19.530718 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.523597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36787 (local address 127.0.0.1:58038)
0619 02:45:19.523805 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.523815 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.523891 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.524022 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.524023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.524347 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:19.524406 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.524409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.525094 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.525095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.525637 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.525639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.525966 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.525967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.526030 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.526523 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:19.526527 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:19.527945 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:19.529534 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.529541 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.529544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.529738 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.530041 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:19.530042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:19.530042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:19.530109 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:19.530370 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:19.530377 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:19.530425 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.530485 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.530598 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":107,"threads_started":1}
I20260619 02:45:19.530726 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.523656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58038 (local address 127.19.113.126:36787)
0619 02:45:19.523748 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.523750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.523883 (+ 133us) server_negotiation.cc:407] Connection header received
0619 02:45:19.523944 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.523945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.523954 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.524014 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:19.524482 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.525011 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.525714 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.525884 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.525963 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.528058 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:19.528059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:19.528059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:19.528066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:19.529429 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.529812 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.529813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.529813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.529864 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:19.530167 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:19.530168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:19.530168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:19.530290 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:19.530356 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.530514 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.530646 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":56}
I20260619 02:45:19.532665 21644 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:58038:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:45:19.532855 21644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:19.534224 21922 tablet_service.cc:1467] Processing CreateTablet for tablet 63991fa504c84439b66ec7c9e8235690 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba4881934e684efea0ac8b245616f0af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:19.534195 21782 tablet_service.cc:1467] Processing CreateTablet for tablet 63991fa504c84439b66ec7c9e8235690 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba4881934e684efea0ac8b245616f0af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:19.534286 22062 tablet_service.cc:1467] Processing CreateTablet for tablet 63991fa504c84439b66ec7c9e8235690 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba4881934e684efea0ac8b245616f0af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:19.534376 21922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63991fa504c84439b66ec7c9e8235690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.534379 21782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63991fa504c84439b66ec7c9e8235690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.534425 22062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63991fa504c84439b66ec7c9e8235690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.535884 22148 tablet_bootstrap.cc:492] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: Bootstrap starting.
I20260619 02:45:19.535912 22149 tablet_bootstrap.cc:492] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: Bootstrap starting.
I20260619 02:45:19.536283 22149 tablet_bootstrap.cc:654] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.536492 22148 tablet_bootstrap.cc:654] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.536849 22152 tablet_bootstrap.cc:492] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: Bootstrap starting.
I20260619 02:45:19.537055 22149 tablet_bootstrap.cc:492] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: No bootstrap required, opened a new log
I20260619 02:45:19.537110 22149 ts_tablet_manager.cc:1397] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:19.537236 22148 tablet_bootstrap.cc:492] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: No bootstrap required, opened a new log
I20260619 02:45:19.537281 22148 ts_tablet_manager.cc:1397] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:19.537238 22149 raft_consensus.cc:348] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.537366 22152 tablet_bootstrap.cc:654] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.537369 22149 raft_consensus.cc:374] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.537401 22149 raft_consensus.cc:729] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28210ebc05944a74a01e05f4f42b24e6, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.537411 22148 raft_consensus.cc:348] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.537482 22148 raft_consensus.cc:374] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.537451 22149 consensus_queue.cc:260] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.537498 22148 raft_consensus.cc:729] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0050f242fb53482db19392387217e533, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.537539 22148 consensus_queue.cc:260] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.537560 22149 ts_tablet_manager.cc:1428] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.537621 22148 ts_tablet_manager.cc:1428] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.537595 22154 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.537664 22154 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:19.537835 22153 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.537907 22153 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:19.538340 22152 tablet_bootstrap.cc:492] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: No bootstrap required, opened a new log
I20260619 02:45:19.538395 22152 ts_tablet_manager.cc:1397] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:45:19.538532 22152 raft_consensus.cc:348] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.538600 22152 raft_consensus.cc:374] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.538612 22152 raft_consensus.cc:729] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee2accedc6d49418052fb60d9e88daa, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.538643 22152 consensus_queue.cc:260] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.538720 22152 ts_tablet_manager.cc:1428] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.538818 22156 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.538925 22156 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:19.541945 22156 raft_consensus.cc:482] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:19.542006 22156 raft_consensus.cc:504] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.542294 22156 leader_election.cc:288] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0050f242fb53482db19392387217e533 (127.19.113.65:46457), 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665)
I20260619 02:45:19.546324 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.542644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:35761 (local address 127.19.113.66:40665)
0619 02:45:19.542704 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.542706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.542714 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:19.542735 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.542736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.542743 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.542791 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.542943 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.543745 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.545307 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.545910 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.545980 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.545990 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.546202 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.546252 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:45:19.546329 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.542480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:54177 (local address 127.19.113.65:46457)
0619 02:45:19.542526 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.542529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.542645 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:45:19.542703 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.542704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.542714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.542777 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.542937 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.543705 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.545306 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.545901 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.545988 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.545996 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.546215 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.546271 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:19.546341 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.542437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:40665 (local address 127.19.113.67:35761)
0619 02:45:19.542481 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.542493 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.542541 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.542803 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.542804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.542805 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.542861 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.542862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.543830 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.543830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.545221 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.545225 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.546007 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.546009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.546117 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.546123 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.546187 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.546247 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":110}
I20260619 02:45:19.546362 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.542451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46457 (local address 127.19.113.67:54177)
0619 02:45:19.542595 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.542604 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.542653 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.542780 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.542780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.542782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.542889 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.542890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.543812 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.543814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.545232 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.545234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.546057 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.546058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.546117 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.546124 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.546187 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.546247 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":35,"thread_start_us":98,"threads_started":1}
I20260619 02:45:19.546690 21942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63991fa504c84439b66ec7c9e8235690" candidate_uuid: "1ee2accedc6d49418052fb60d9e88daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28210ebc05944a74a01e05f4f42b24e6" is_pre_election: true
I20260619 02:45:19.546804 21942 raft_consensus.cc:2454] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ee2accedc6d49418052fb60d9e88daa in term 0.
I20260619 02:45:19.546924 21802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63991fa504c84439b66ec7c9e8235690" candidate_uuid: "1ee2accedc6d49418052fb60d9e88daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0050f242fb53482db19392387217e533" is_pre_election: true
I20260619 02:45:19.547003 21802 raft_consensus.cc:2454] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ee2accedc6d49418052fb60d9e88daa in term 0.
I20260619 02:45:19.547158 22016 leader_election.cc:302] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0050f242fb53482db19392387217e533, 1ee2accedc6d49418052fb60d9e88daa; no voters:
I20260619 02:45:19.547307 22156 raft_consensus.cc:2781] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:19.547358 22156 raft_consensus.cc:482] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:19.547374 22156 raft_consensus.cc:3037] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.548049 22156 raft_consensus.cc:504] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.548185 22156 leader_election.cc:288] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [CANDIDATE]: Term 1 election: Requested vote from peers 0050f242fb53482db19392387217e533 (127.19.113.65:46457), 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665)
I20260619 02:45:19.548338 21802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63991fa504c84439b66ec7c9e8235690" candidate_uuid: "1ee2accedc6d49418052fb60d9e88daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0050f242fb53482db19392387217e533"
I20260619 02:45:19.548367 21942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63991fa504c84439b66ec7c9e8235690" candidate_uuid: "1ee2accedc6d49418052fb60d9e88daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28210ebc05944a74a01e05f4f42b24e6"
W20260619 02:45:19.548389 21849 tablet.cc:2369] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:19.548426 21942 raft_consensus.cc:3037] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.548426 21802 raft_consensus.cc:3037] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.549220 21942 raft_consensus.cc:2454] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee2accedc6d49418052fb60d9e88daa in term 1.
I20260619 02:45:19.549220 21802 raft_consensus.cc:2454] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee2accedc6d49418052fb60d9e88daa in term 1.
I20260619 02:45:19.549369 22016 leader_election.cc:302] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0050f242fb53482db19392387217e533, 1ee2accedc6d49418052fb60d9e88daa; no voters:
I20260619 02:45:19.549455 22156 raft_consensus.cc:2781] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:19.549625 22156 raft_consensus.cc:686] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [term 1 LEADER]: Becoming Leader. State: Replica: 1ee2accedc6d49418052fb60d9e88daa, State: Running, Role: LEADER
I20260619 02:45:19.549728 22156 consensus_queue.cc:237] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.550223 22170 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: TxnStatusTablet state changed. Reason: New leader 1ee2accedc6d49418052fb60d9e88daa. Latest consensus state: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.550309 22170 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:19.550478 22172 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:45:19.550578 22172 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:45:19.550671 21644 catalog_manager.cc:5495] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67). New cstate: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } health_report { overall_health: HEALTHY } } }
I20260619 02:45:19.561317 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.558432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40499 (local address 127.0.0.1:37474)
0619 02:45:19.558500 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.558511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.558571 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.558821 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.558822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.558823 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:19.558894 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.558897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.559817 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.559817 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.560388 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.560389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.560849 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.560850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.560902 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.560917 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.561182 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.561187 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.561217 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.561259 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:45:19.561334 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.558640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.19.113.67:40499)
0619 02:45:19.558696 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.558697 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.558704 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:19.558738 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.558738 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.558753 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.558806 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:19.558959 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.559744 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.560455 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.560771 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.560831 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.560990 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.561125 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.561201 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.561222 (+ 21us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.561241 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:45:19.563143 21802 raft_consensus.cc:1261] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Refusing update from remote peer 1ee2accedc6d49418052fb60d9e88daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:19.563169 21942 raft_consensus.cc:1261] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Refusing update from remote peer 1ee2accedc6d49418052fb60d9e88daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:19.563463 22170 consensus_queue.cc:1035] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:19.563688 22156 consensus_queue.cc:1035] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:19.564626 22153 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: TxnStatusTablet state changed. Reason: New leader 1ee2accedc6d49418052fb60d9e88daa. Latest consensus state: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.564738 22153 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:19.564824 22154 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: TxnStatusTablet state changed. Reason: New leader 1ee2accedc6d49418052fb60d9e88daa. Latest consensus state: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.564911 22154 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:19.565827 22156 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.565909 22156 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:19.565979 22153 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.566071 22153 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 0050f242fb53482db19392387217e533: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:19.566087 22176 mvcc.cc:205] Tried to move back new op lower bound from 7298404841727971328 to 7298404841676378112. Current Snapshot: MvccSnapshot[applied={T|T < 7298404841727971328}]
I20260619 02:45:19.566367 22170 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.566444 22170 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 1ee2accedc6d49418052fb60d9e88daa: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:19.566905 22154 tablet_replica.cc:439] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } } }
I20260619 02:45:19.566987 22154 tablet_replica.cc:442] T 63991fa504c84439b66ec7c9e8235690 P 28210ebc05944a74a01e05f4f42b24e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:19.567750 22178 mvcc.cc:205] Tried to move back new op lower bound from 7298404841727971328 to 7298404841676378112. Current Snapshot: MvccSnapshot[applied={T|T < 7298404841727971328 or (T in {7298404841727971328})}]
I20260619 02:45:19.571322 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.568668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.19.113.66:40665)
0619 02:45:19.568706 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.568708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.568716 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:19.568773 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.568774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.568790 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.568865 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:19.568962 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.569690 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.570420 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.570751 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.570822 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.571014 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.571131 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.571209 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.571229 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.571251 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:45:19.576498 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.573826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:40031 (local address 127.19.113.67:40499)
0619 02:45:19.573892 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.573894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.573903 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:19.573959 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.573960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.573970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.574037 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:19.574165 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.574932 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.575741 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.576068 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.576135 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.576281 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.576314 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.576360 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.576400 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.576420 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:45:19.576525 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.573775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40499 (local address 127.19.113.66:40031)
0619 02:45:19.573858 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.573869 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.573900 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.574051 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.574052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.574053 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:19.574109 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.574110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.575061 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.575063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.575662 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.575664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.576135 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.576136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.576186 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.576200 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.576364 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.576365 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.576399 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.576443 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 02:45:19.578837 21801 raft_consensus.cc:1261] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Refusing update from remote peer 28210ebc05944a74a01e05f4f42b24e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:19.578958 22082 raft_consensus.cc:1261] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Refusing update from remote peer 28210ebc05944a74a01e05f4f42b24e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:19.579131 22158 consensus_queue.cc:1035] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:19.579288 22154 consensus_queue.cc:1035] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:19.582038 22175 mvcc.cc:205] Tried to move back new op lower bound from 7298404841793191936 to 7298404841455149056. Current Snapshot: MvccSnapshot[applied={T|T < 7298404841793191936}]
I20260619 02:45:19.588770 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.585931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:40665 (local address 127.19.113.67:58837)
0619 02:45:19.586012 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.586024 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.586067 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.586221 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.586222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.586224 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:19.586290 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.586291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.586951 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.586952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.587961 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.587965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.588337 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.588338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.588408 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.588425 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.588609 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.588612 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.588653 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.588701 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 02:45:19.588770 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.585976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:58837 (local address 127.19.113.66:40665)
0619 02:45:19.586012 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.586014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.586062 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:45:19.586109 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.586109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.586119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.586174 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:19.586345 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.586846 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.588067 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.588253 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.588326 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.588505 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.588540 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.588584 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.588667 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.588690 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:45:19.599870 21644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:19.600004 21644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:19.600857 21644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: Sending DeleteTablet for 3 replicas of tablet e2f3d2e2a764434db39985d68aaf686f
I20260619 02:45:19.601220 22062 tablet_service.cc:1514] Processing DeleteTablet for tablet e2f3d2e2a764434db39985d68aaf686f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:19 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:37462
I20260619 02:45:19.601506 22193 tablet_replica.cc:330] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: stopping tablet replica
I20260619 02:45:19.601600 21922 tablet_service.cc:1514] Processing DeleteTablet for tablet e2f3d2e2a764434db39985d68aaf686f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:19 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:51938
I20260619 02:45:19.601783 22193 raft_consensus.cc:2229] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:19.601893 22193 raft_consensus.cc:2258] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:19.602209 21782 tablet_service.cc:1514] Processing DeleteTablet for tablet e2f3d2e2a764434db39985d68aaf686f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:19 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:40640
I20260619 02:45:19.602438 22193 ts_tablet_manager.cc:1905] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:19.602686 21644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:19.602846 21644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:19.604081 22062 tablet_service.cc:1467] Processing CreateTablet for tablet e0f2481c653a478f8d8ebe088b00b683 (DEFAULT_TABLE table=test-table [id=0098e40de36c4b3aaada0b8d23217b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:19.604205 22062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f2481c653a478f8d8ebe088b00b683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.604633 22193 ts_tablet_manager.cc:1918] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:19.604768 22193 log.cc:1196] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/e2f3d2e2a764434db39985d68aaf686f
I20260619 02:45:19.605079 22193 ts_tablet_manager.cc:1939] T e2f3d2e2a764434db39985d68aaf686f P 1ee2accedc6d49418052fb60d9e88daa: Deleting consensus metadata
I20260619 02:45:19.605471 22152 tablet_bootstrap.cc:492] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: Bootstrap starting.
I20260619 02:45:19.605597 21629 catalog_manager.cc:4841] TS 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499): tablet e2f3d2e2a764434db39985d68aaf686f (table test-table [id=e601f919272a44be9db9b956bac45b51]) successfully deleted
I20260619 02:45:19.605859 22152 tablet_bootstrap.cc:654] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.605861 21922 tablet_service.cc:1467] Processing CreateTablet for tablet e0f2481c653a478f8d8ebe088b00b683 (DEFAULT_TABLE table=test-table [id=0098e40de36c4b3aaada0b8d23217b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:19.605988 21922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f2481c653a478f8d8ebe088b00b683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.606444 22152 tablet_bootstrap.cc:492] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: No bootstrap required, opened a new log
I20260619 02:45:19.606488 22152 ts_tablet_manager.cc:1397] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:19.606627 22152 raft_consensus.cc:348] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.606688 22152 raft_consensus.cc:374] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.606709 22152 raft_consensus.cc:729] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee2accedc6d49418052fb60d9e88daa, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.606750 22152 consensus_queue.cc:260] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.606837 22152 ts_tablet_manager.cc:1428] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.607271 22149 tablet_bootstrap.cc:492] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: Bootstrap starting.
I20260619 02:45:19.607654 22149 tablet_bootstrap.cc:654] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.608296 21782 tablet_service.cc:1467] Processing CreateTablet for tablet e0f2481c653a478f8d8ebe088b00b683 (DEFAULT_TABLE table=test-table [id=0098e40de36c4b3aaada0b8d23217b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:19.608351 22149 tablet_bootstrap.cc:492] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: No bootstrap required, opened a new log
I20260619 02:45:19.608400 22149 ts_tablet_manager.cc:1397] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:19.608426 21782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f2481c653a478f8d8ebe088b00b683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:19.608541 22149 raft_consensus.cc:348] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.608609 22149 raft_consensus.cc:374] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.608628 22149 raft_consensus.cc:729] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28210ebc05944a74a01e05f4f42b24e6, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.608666 22149 consensus_queue.cc:260] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.608743 22149 ts_tablet_manager.cc:1428] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 02:45:19.609366 22129 tablet.cc:2369] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:19.609689 22148 tablet_bootstrap.cc:492] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: Bootstrap starting.
I20260619 02:45:19.610087 22148 tablet_bootstrap.cc:654] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:19.610901 22148 tablet_bootstrap.cc:492] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: No bootstrap required, opened a new log
I20260619 02:45:19.610968 22148 ts_tablet_manager.cc:1397] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:19.611112 22148 raft_consensus.cc:348] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.611181 22148 raft_consensus.cc:374] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:19.611199 22148 raft_consensus.cc:729] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0050f242fb53482db19392387217e533, State: Initialized, Role: FOLLOWER
I20260619 02:45:19.611238 22148 consensus_queue.cc:260] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.611311 22148 ts_tablet_manager.cc:1428] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:19.612452 22195 tablet_replica.cc:330] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: stopping tablet replica
I20260619 02:45:19.612572 22195 raft_consensus.cc:2229] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:19.612643 22195 raft_consensus.cc:2258] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:19.612807 22194 tablet_replica.cc:330] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: stopping tablet replica
I20260619 02:45:19.612914 22194 raft_consensus.cc:2229] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:19.613018 22194 raft_consensus.cc:2258] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:19.613724 22195 ts_tablet_manager.cc:1905] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:19.615357 22194 ts_tablet_manager.cc:1905] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:19.615608 22195 ts_tablet_manager.cc:1918] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:19.615693 22195 log.cc:1196] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/e2f3d2e2a764434db39985d68aaf686f
I20260619 02:45:19.615955 22195 ts_tablet_manager.cc:1939] T e2f3d2e2a764434db39985d68aaf686f P 0050f242fb53482db19392387217e533: Deleting consensus metadata
I20260619 02:45:19.616355 21632 catalog_manager.cc:4841] TS 0050f242fb53482db19392387217e533 (127.19.113.65:46457): tablet e2f3d2e2a764434db39985d68aaf686f (table test-table [id=e601f919272a44be9db9b956bac45b51]) successfully deleted
I20260619 02:45:19.617412 22194 ts_tablet_manager.cc:1918] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:19.617484 22194 log.cc:1196] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/e2f3d2e2a764434db39985d68aaf686f
I20260619 02:45:19.617765 22194 ts_tablet_manager.cc:1939] T e2f3d2e2a764434db39985d68aaf686f P 28210ebc05944a74a01e05f4f42b24e6: Deleting consensus metadata
I20260619 02:45:19.618125 21629 catalog_manager.cc:4841] TS 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665): tablet e2f3d2e2a764434db39985d68aaf686f (table test-table [id=e601f919272a44be9db9b956bac45b51]) successfully deleted
I20260619 02:45:19.618538 22153 raft_consensus.cc:482] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:19.618595 22153 raft_consensus.cc:504] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.618892 22153 leader_election.cc:288] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665), 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499)
I20260619 02:45:19.622969 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.619331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:42299 (local address 127.19.113.67:40499)
0619 02:45:19.619376 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.619378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.619603 (+ 225us) server_negotiation.cc:407] Connection header received
0619 02:45:19.619692 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.619693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.619702 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.619727 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.619908 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.620616 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.622015 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.622571 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.622617 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.622626 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.622822 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.622861 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:45:19.623111 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.619324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:42881 (local address 127.19.113.66:40665)
0619 02:45:19.619365 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.619367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.619374 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:19.619396 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.619397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.619417 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.619473 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:19.619652 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.620420 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.622081 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.622629 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.622672 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.622680 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.622929 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.623044 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:19.623214 22200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.619184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40499 (local address 127.19.113.65:42299)
0619 02:45:19.619543 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.619553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.619627 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.619780 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.619780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.619782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.619846 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.619847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.620700 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.620701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.621912 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.621916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.622682 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.622683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.622754 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.622758 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.623084 (+ 326us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.623130 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":35,"thread_start_us":29,"threads_started":1}
I20260619 02:45:19.623425 22082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f2481c653a478f8d8ebe088b00b683" candidate_uuid: "0050f242fb53482db19392387217e533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee2accedc6d49418052fb60d9e88daa" is_pre_election: true
I20260619 02:45:19.623517 22082 raft_consensus.cc:2454] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0050f242fb53482db19392387217e533 in term 0.
I20260619 02:45:19.623718 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.619049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:40665 (local address 127.19.113.65:42881)
0619 02:45:19.619140 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.619153 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.619208 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.619492 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.619493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.619495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:19.619565 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.619567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.620759 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.620760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.621980 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.621983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.622734 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.622735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.622823 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.622827 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.623589 (+ 762us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.623634 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":51,"threads_started":1}
I20260619 02:45:19.623696 21733 leader_election.cc:302] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0050f242fb53482db19392387217e533, 1ee2accedc6d49418052fb60d9e88daa; no voters:
I20260619 02:45:19.624370 22153 raft_consensus.cc:2781] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:19.624420 22153 raft_consensus.cc:482] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:19.624454 22153 raft_consensus.cc:3037] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.624408 21942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f2481c653a478f8d8ebe088b00b683" candidate_uuid: "0050f242fb53482db19392387217e533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28210ebc05944a74a01e05f4f42b24e6" is_pre_election: true
I20260619 02:45:19.624483 21942 raft_consensus.cc:2454] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0050f242fb53482db19392387217e533 in term 0.
I20260619 02:45:19.625255 22153 raft_consensus.cc:504] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.625396 22153 leader_election.cc:288] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [CANDIDATE]: Term 1 election: Requested vote from peers 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665), 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499)
I20260619 02:45:19.625571 22082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f2481c653a478f8d8ebe088b00b683" candidate_uuid: "0050f242fb53482db19392387217e533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee2accedc6d49418052fb60d9e88daa"
I20260619 02:45:19.625586 21942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f2481c653a478f8d8ebe088b00b683" candidate_uuid: "0050f242fb53482db19392387217e533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28210ebc05944a74a01e05f4f42b24e6"
I20260619 02:45:19.625634 22082 raft_consensus.cc:3037] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.625649 21942 raft_consensus.cc:3037] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:19.626305 22082 raft_consensus.cc:2454] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0050f242fb53482db19392387217e533 in term 1.
I20260619 02:45:19.626394 21942 raft_consensus.cc:2454] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0050f242fb53482db19392387217e533 in term 1.
I20260619 02:45:19.626463 21733 leader_election.cc:302] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0050f242fb53482db19392387217e533, 1ee2accedc6d49418052fb60d9e88daa; no voters:
I20260619 02:45:19.626560 22153 raft_consensus.cc:2781] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:19.626700 22153 raft_consensus.cc:686] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 1 LEADER]: Becoming Leader. State: Replica: 0050f242fb53482db19392387217e533, State: Running, Role: LEADER
I20260619 02:45:19.626765 22153 consensus_queue.cc:237] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:19.627477 21644 catalog_manager.cc:5495] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0050f242fb53482db19392387217e533 (127.19.113.65). New cstate: current_term: 1 leader_uuid: "0050f242fb53482db19392387217e533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } health_report { overall_health: UNKNOWN } } }
I20260619 02:45:19.640451 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.637378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.19.113.65:46457)
0619 02:45:19.637432 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.637434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.637442 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:19.637477 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.637478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.637500 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.637572 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:19.637708 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.638460 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.639546 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.639850 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.639912 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.640081 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.640211 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.640287 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.640344 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.640370 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:45:19.646502 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.643260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:41683 (local address 127.19.113.67:40499)
0619 02:45:19.643306 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.643308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.643410 (+ 102us) server_negotiation.cc:407] Connection header received
0619 02:45:19.643464 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.643465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.643475 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.643535 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:19.643687 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.644441 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.645647 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.645967 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.646041 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.646246 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.646283 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.646334 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.646389 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.646411 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:19.646909 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.643193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40499 (local address 127.19.113.65:41683)
0619 02:45:19.643344 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.643354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.643411 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.643535 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.643536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.643537 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:19.643625 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.643626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.644519 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.644521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.645557 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.645561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.646057 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.646059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.646123 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.646141 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.646341 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.646343 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.646385 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.646426 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260619 02:45:19.649401 22082 raft_consensus.cc:1261] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Refusing update from remote peer 0050f242fb53482db19392387217e533: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:19.649437 21942 raft_consensus.cc:1261] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Refusing update from remote peer 0050f242fb53482db19392387217e533: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:19.649703 22153 consensus_queue.cc:1035] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:19.649861 22201 consensus_queue.cc:1035] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:19.659538 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.655899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46457 (local address 127.19.113.67:37699)
0619 02:45:19.655990 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:19.656000 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:19.656035 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:19.656598 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:19.656599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:19.656600 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:19.656649 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.656650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.657732 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.657733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.658596 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:19.658599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.659138 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.659139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:19.659196 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.659201 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.659370 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.659372 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:45:19.659407 (+ 35us) client_negotiation.cc:240] Negotiation successful
0619 02:45:19.659452 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 02:45:19.659992 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:19.656120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:37699 (local address 127.19.113.65:46457)
0619 02:45:19.656156 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:19.656158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:19.656164 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:45:19.656183 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:19.656184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:19.656192 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:19.656232 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:19.656726 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.657482 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.658689 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:19.659048 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:19.659106 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:19.659266 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:19.659313 (+ 47us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:19.659350 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:19.659895 (+ 545us) server_negotiation.cc:299] Negotiation successful
0619 02:45:19.659917 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:45:19.660470 22176 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:45:21.185636 21644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:21.185747 21644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:21.186550 21644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: Sending DeleteTablet for 3 replicas of tablet e0f2481c653a478f8d8ebe088b00b683
I20260619 02:45:21.186822 21922 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f2481c653a478f8d8ebe088b00b683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:21 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:51938
I20260619 02:45:21.186826 21782 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f2481c653a478f8d8ebe088b00b683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:21 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:40640
I20260619 02:45:21.186859 22062 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f2481c653a478f8d8ebe088b00b683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:21 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:37462
I20260619 02:45:21.187136 22231 tablet_replica.cc:330] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: stopping tablet replica
I20260619 02:45:21.187188 22231 raft_consensus.cc:2229] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:21.187218 22231 raft_consensus.cc:2258] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:21.187302 22232 tablet_replica.cc:330] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: stopping tablet replica
I20260619 02:45:21.187350 22232 raft_consensus.cc:2229] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:21.187381 22232 raft_consensus.cc:2258] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:21.187597 22231 ts_tablet_manager.cc:1905] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:21.189508 22231 ts_tablet_manager.cc:1918] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:45:21.189561 22231 log.cc:1196] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/e0f2481c653a478f8d8ebe088b00b683
I20260619 02:45:21.189735 22231 ts_tablet_manager.cc:1939] T e0f2481c653a478f8d8ebe088b00b683 P 28210ebc05944a74a01e05f4f42b24e6: Deleting consensus metadata
I20260619 02:45:21.189965 22232 ts_tablet_manager.cc:1905] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:21.190284 21629 catalog_manager.cc:4841] TS 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665): tablet e0f2481c653a478f8d8ebe088b00b683 (table test-table [id=0098e40de36c4b3aaada0b8d23217b1f]) successfully deleted
I20260619 02:45:21.191444 22232 ts_tablet_manager.cc:1918] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:45:21.191497 22232 log.cc:1196] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/e0f2481c653a478f8d8ebe088b00b683
I20260619 02:45:21.191661 22232 ts_tablet_manager.cc:1939] T e0f2481c653a478f8d8ebe088b00b683 P 1ee2accedc6d49418052fb60d9e88daa: Deleting consensus metadata
I20260619 02:45:21.191812 22233 tablet_replica.cc:330] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: stopping tablet replica
I20260619 02:45:21.191874 22233 raft_consensus.cc:2229] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:21.191984 22233 raft_consensus.cc:2258] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:21.192114 21629 catalog_manager.cc:4841] TS 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499): tablet e0f2481c653a478f8d8ebe088b00b683 (table test-table [id=0098e40de36c4b3aaada0b8d23217b1f]) successfully deleted
I20260619 02:45:21.192308 22233 ts_tablet_manager.cc:1905] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:21.193728 22233 ts_tablet_manager.cc:1918] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:45:21.193779 22233 log.cc:1196] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/e0f2481c653a478f8d8ebe088b00b683
I20260619 02:45:21.193964 22233 ts_tablet_manager.cc:1939] T e0f2481c653a478f8d8ebe088b00b683 P 0050f242fb53482db19392387217e533: Deleting consensus metadata
I20260619 02:45:21.194489 21632 catalog_manager.cc:4841] TS 0050f242fb53482db19392387217e533 (127.19.113.65:46457): tablet e0f2481c653a478f8d8ebe088b00b683 (table test-table [id=0098e40de36c4b3aaada0b8d23217b1f]) successfully deleted
Jun 19 02:45:21 dist-test-slave-zqpb krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837121, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:45:21 dist-test-slave-zqpb krb5kdc[21600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837121, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:21.211370 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:21.203790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.19.113.126:36787)
0619 02:45:21.203916 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:21.203918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:21.203979 (+ 61us) server_negotiation.cc:407] Connection header received
0619 02:45:21.204049 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:21.204050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:21.204063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:21.204125 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:21.204728 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.205263 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.206095 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.206309 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.206365 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:21.208744 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:21.208746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:21.208746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:21.208754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:21.210198 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:21.210545 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:21.210547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:21.210548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:21.210643 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:21.210811 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:21.210812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:21.210812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:21.210974 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:21.211070 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:21.211183 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 02:45:21.211268 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260619 02:45:21.213084 21644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:21.213238 21644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:21.214613 21782 tablet_service.cc:1467] Processing CreateTablet for tablet 1466c707561545f295f7e395083e1add (DEFAULT_TABLE table=test-table [id=a9d0dcf6d54b466b9e6e97fa2fad29d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:21.214625 22062 tablet_service.cc:1467] Processing CreateTablet for tablet 1466c707561545f295f7e395083e1add (DEFAULT_TABLE table=test-table [id=a9d0dcf6d54b466b9e6e97fa2fad29d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:21.214742 21782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1466c707561545f295f7e395083e1add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:21.214746 22062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1466c707561545f295f7e395083e1add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:21.214973 21922 tablet_service.cc:1467] Processing CreateTablet for tablet 1466c707561545f295f7e395083e1add (DEFAULT_TABLE table=test-table [id=a9d0dcf6d54b466b9e6e97fa2fad29d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:21.215092 21922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1466c707561545f295f7e395083e1add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:21.216200 22243 tablet_bootstrap.cc:492] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: Bootstrap starting.
I20260619 02:45:21.216455 22244 tablet_bootstrap.cc:492] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: Bootstrap starting.
I20260619 02:45:21.216575 22242 tablet_bootstrap.cc:492] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: Bootstrap starting.
I20260619 02:45:21.216639 22243 tablet_bootstrap.cc:654] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:21.216903 22244 tablet_bootstrap.cc:654] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:21.216936 22242 tablet_bootstrap.cc:654] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:21.217268 22243 tablet_bootstrap.cc:492] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: No bootstrap required, opened a new log
I20260619 02:45:21.217325 22243 ts_tablet_manager.cc:1397] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:21.217509 22242 tablet_bootstrap.cc:492] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: No bootstrap required, opened a new log
I20260619 02:45:21.217559 22242 ts_tablet_manager.cc:1397] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:21.217479 22243 raft_consensus.cc:348] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.217681 22244 tablet_bootstrap.cc:492] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: No bootstrap required, opened a new log
I20260619 02:45:21.217701 22242 raft_consensus.cc:348] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.217772 22244 ts_tablet_manager.cc:1397] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:21.217851 22242 raft_consensus.cc:374] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:21.217885 22243 raft_consensus.cc:374] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:21.217908 22242 raft_consensus.cc:729] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee2accedc6d49418052fb60d9e88daa, State: Initialized, Role: FOLLOWER
I20260619 02:45:21.217958 22243 raft_consensus.cc:729] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0050f242fb53482db19392387217e533, State: Initialized, Role: FOLLOWER
I20260619 02:45:21.217933 22244 raft_consensus.cc:348] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.217974 22242 consensus_queue.cc:260] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.218034 22244 raft_consensus.cc:374] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:21.218040 22243 consensus_queue.cc:260] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.218097 22244 raft_consensus.cc:729] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28210ebc05944a74a01e05f4f42b24e6, State: Initialized, Role: FOLLOWER
I20260619 02:45:21.218117 22242 ts_tablet_manager.cc:1428] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:21.218186 22244 consensus_queue.cc:260] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.218358 22244 ts_tablet_manager.cc:1428] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:21.218538 22243 ts_tablet_manager.cc:1428] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:21.447925 22153 raft_consensus.cc:482] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:21.448074 22153 raft_consensus.cc:504] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.448204 22153 leader_election.cc:288] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665), 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499)
I20260619 02:45:21.448434 21942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1466c707561545f295f7e395083e1add" candidate_uuid: "0050f242fb53482db19392387217e533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28210ebc05944a74a01e05f4f42b24e6" is_pre_election: true
I20260619 02:45:21.448462 22082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1466c707561545f295f7e395083e1add" candidate_uuid: "0050f242fb53482db19392387217e533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee2accedc6d49418052fb60d9e88daa" is_pre_election: true
I20260619 02:45:21.448541 22082 raft_consensus.cc:2454] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0050f242fb53482db19392387217e533 in term 0.
I20260619 02:45:21.448541 21942 raft_consensus.cc:2454] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0050f242fb53482db19392387217e533 in term 0.
I20260619 02:45:21.448709 21733 leader_election.cc:302] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0050f242fb53482db19392387217e533, 1ee2accedc6d49418052fb60d9e88daa; no voters:
I20260619 02:45:21.448797 22153 raft_consensus.cc:2781] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:21.448836 22153 raft_consensus.cc:482] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:21.448859 22153 raft_consensus.cc:3037] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:21.449496 22153 raft_consensus.cc:504] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.449606 22153 leader_election.cc:288] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [CANDIDATE]: Term 1 election: Requested vote from peers 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665), 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499)
I20260619 02:45:21.449779 22082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1466c707561545f295f7e395083e1add" candidate_uuid: "0050f242fb53482db19392387217e533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee2accedc6d49418052fb60d9e88daa"
I20260619 02:45:21.449772 21942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1466c707561545f295f7e395083e1add" candidate_uuid: "0050f242fb53482db19392387217e533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28210ebc05944a74a01e05f4f42b24e6"
I20260619 02:45:21.449851 22082 raft_consensus.cc:3037] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:21.449851 21942 raft_consensus.cc:3037] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:21.450614 22082 raft_consensus.cc:2454] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0050f242fb53482db19392387217e533 in term 1.
I20260619 02:45:21.450614 21942 raft_consensus.cc:2454] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0050f242fb53482db19392387217e533 in term 1.
I20260619 02:45:21.450775 21733 leader_election.cc:302] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0050f242fb53482db19392387217e533, 28210ebc05944a74a01e05f4f42b24e6; no voters:
I20260619 02:45:21.450898 22153 raft_consensus.cc:2781] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:21.450951 22153 raft_consensus.cc:686] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 1 LEADER]: Becoming Leader. State: Replica: 0050f242fb53482db19392387217e533, State: Running, Role: LEADER
I20260619 02:45:21.451009 22153 consensus_queue.cc:237] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.451596 21644 catalog_manager.cc:5495] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0050f242fb53482db19392387217e533 (127.19.113.65). New cstate: current_term: 1 leader_uuid: "0050f242fb53482db19392387217e533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } health_report { overall_health: UNKNOWN } } }
I20260619 02:45:21.503326 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:21.500975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40652 (local address 127.19.113.65:46457)
0619 02:45:21.501115 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:21.501117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:21.501124 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:21.501145 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:21.501146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:21.501157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:21.501235 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:21.501347 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.501859 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.502680 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.502912 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.502965 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:21.503095 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:21.503130 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:21.503171 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:21.503220 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 02:45:21.503241 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260619 02:45:21.505359 22082 raft_consensus.cc:1261] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Refusing update from remote peer 0050f242fb53482db19392387217e533: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:21.505405 21942 raft_consensus.cc:1261] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Refusing update from remote peer 0050f242fb53482db19392387217e533: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:21.505620 22222 consensus_queue.cc:1035] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:21.505744 22153 consensus_queue.cc:1035] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:21.515174 21644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:21.515278 21644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:21.515852 21644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: Sending DeleteTablet for 3 replicas of tablet 1466c707561545f295f7e395083e1add
I20260619 02:45:21.516065 21782 tablet_service.cc:1514] Processing DeleteTablet for tablet 1466c707561545f295f7e395083e1add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:21 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:40640
I20260619 02:45:21.516065 21922 tablet_service.cc:1514] Processing DeleteTablet for tablet 1466c707561545f295f7e395083e1add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:21 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:51938
I20260619 02:45:21.516157 22231 tablet_replica.cc:330] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: stopping tablet replica
I20260619 02:45:21.516211 22231 raft_consensus.cc:2229] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:21.516216 22062 tablet_service.cc:1514] Processing DeleteTablet for tablet 1466c707561545f295f7e395083e1add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:21 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:37462
I20260619 02:45:21.516244 22231 raft_consensus.cc:2258] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:21.516305 22233 tablet_replica.cc:330] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: stopping tablet replica
I20260619 02:45:21.516307 22232 tablet_replica.cc:330] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: stopping tablet replica
I20260619 02:45:21.516350 22233 raft_consensus.cc:2229] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:21.516367 22232 raft_consensus.cc:2229] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:21.516402 22232 raft_consensus.cc:2258] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:21.516396 22233 raft_consensus.cc:2258] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:21.516340 21644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:21.516466 21644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:21.516584 22231 ts_tablet_manager.cc:1905] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:21.516825 22233 ts_tablet_manager.cc:1905] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:21.516867 22232 ts_tablet_manager.cc:1905] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:21.517717 22062 tablet_service.cc:1467] Processing CreateTablet for tablet e09ab0007b66446884373dbcebea8f35 (DEFAULT_TABLE table=test-table [id=8e69121ea69242c2beb95d158f40867f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:21.517812 22062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09ab0007b66446884373dbcebea8f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:21.518225 22231 ts_tablet_manager.cc:1918] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:21.518260 22233 ts_tablet_manager.cc:1918] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:21.518277 22231 log.cc:1196] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/1466c707561545f295f7e395083e1add
I20260619 02:45:21.518309 22233 log.cc:1196] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/1466c707561545f295f7e395083e1add
I20260619 02:45:21.518358 22232 ts_tablet_manager.cc:1918] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:21.518399 22232 log.cc:1196] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/1466c707561545f295f7e395083e1add
I20260619 02:45:21.518481 22231 ts_tablet_manager.cc:1939] T 1466c707561545f295f7e395083e1add P 28210ebc05944a74a01e05f4f42b24e6: Deleting consensus metadata
I20260619 02:45:21.518520 22233 ts_tablet_manager.cc:1939] T 1466c707561545f295f7e395083e1add P 0050f242fb53482db19392387217e533: Deleting consensus metadata
I20260619 02:45:21.518577 22232 ts_tablet_manager.cc:1939] T 1466c707561545f295f7e395083e1add P 1ee2accedc6d49418052fb60d9e88daa: Deleting consensus metadata
I20260619 02:45:21.518744 21782 tablet_service.cc:1467] Processing CreateTablet for tablet e09ab0007b66446884373dbcebea8f35 (DEFAULT_TABLE table=test-table [id=8e69121ea69242c2beb95d158f40867f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:21.518827 21782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09ab0007b66446884373dbcebea8f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:21.519165 21632 catalog_manager.cc:4841] TS 0050f242fb53482db19392387217e533 (127.19.113.65:46457): tablet 1466c707561545f295f7e395083e1add (table test-table [id=a9d0dcf6d54b466b9e6e97fa2fad29d3]) successfully deleted
I20260619 02:45:21.519270 21629 catalog_manager.cc:4841] TS 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665): tablet 1466c707561545f295f7e395083e1add (table test-table [id=a9d0dcf6d54b466b9e6e97fa2fad29d3]) successfully deleted
I20260619 02:45:21.519318 22242 tablet_bootstrap.cc:492] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: Bootstrap starting.
I20260619 02:45:21.519315 21922 tablet_service.cc:1467] Processing CreateTablet for tablet e09ab0007b66446884373dbcebea8f35 (DEFAULT_TABLE table=test-table [id=8e69121ea69242c2beb95d158f40867f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:21.519402 21922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09ab0007b66446884373dbcebea8f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:21.519641 22242 tablet_bootstrap.cc:654] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:21.519793 22243 tablet_bootstrap.cc:492] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533: Bootstrap starting.
I20260619 02:45:21.520097 22243 tablet_bootstrap.cc:654] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:21.520125 21629 catalog_manager.cc:4841] TS 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499): tablet 1466c707561545f295f7e395083e1add (table test-table [id=a9d0dcf6d54b466b9e6e97fa2fad29d3]) successfully deleted
I20260619 02:45:21.520184 22242 tablet_bootstrap.cc:492] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: No bootstrap required, opened a new log
I20260619 02:45:21.520227 22242 ts_tablet_manager.cc:1397] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:21.520321 22242 raft_consensus.cc:348] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.520375 22242 raft_consensus.cc:374] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:21.520392 22242 raft_consensus.cc:729] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee2accedc6d49418052fb60d9e88daa, State: Initialized, Role: FOLLOWER
I20260619 02:45:21.520439 22242 consensus_queue.cc:260] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.520529 22242 ts_tablet_manager.cc:1428] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:21.520601 22243 tablet_bootstrap.cc:492] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533: No bootstrap required, opened a new log
I20260619 02:45:21.520648 22243 ts_tablet_manager.cc:1397] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:21.520651 22244 tablet_bootstrap.cc:492] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: Bootstrap starting.
I20260619 02:45:21.520740 22243 raft_consensus.cc:348] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.520799 22243 raft_consensus.cc:374] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:21.520816 22243 raft_consensus.cc:729] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0050f242fb53482db19392387217e533, State: Initialized, Role: FOLLOWER
I20260619 02:45:21.520856 22243 consensus_queue.cc:260] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.520955 22244 tablet_bootstrap.cc:654] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:21.520987 22243 ts_tablet_manager.cc:1428] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:21.521502 22244 tablet_bootstrap.cc:492] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: No bootstrap required, opened a new log
I20260619 02:45:21.521549 22244 ts_tablet_manager.cc:1397] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:21.521661 22244 raft_consensus.cc:348] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.521727 22244 raft_consensus.cc:374] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:21.521747 22244 raft_consensus.cc:729] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28210ebc05944a74a01e05f4f42b24e6, State: Initialized, Role: FOLLOWER
I20260619 02:45:21.521785 22244 consensus_queue.cc:260] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.521869 22244 ts_tablet_manager.cc:1428] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:21.531993 22217 raft_consensus.cc:482] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:21.532086 22217 raft_consensus.cc:504] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.532235 22217 leader_election.cc:288] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0050f242fb53482db19392387217e533 (127.19.113.65:46457), 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665)
I20260619 02:45:21.532389 21801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09ab0007b66446884373dbcebea8f35" candidate_uuid: "1ee2accedc6d49418052fb60d9e88daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0050f242fb53482db19392387217e533" is_pre_election: true
I20260619 02:45:21.532402 21942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09ab0007b66446884373dbcebea8f35" candidate_uuid: "1ee2accedc6d49418052fb60d9e88daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28210ebc05944a74a01e05f4f42b24e6" is_pre_election: true
I20260619 02:45:21.532493 21942 raft_consensus.cc:2454] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ee2accedc6d49418052fb60d9e88daa in term 0.
I20260619 02:45:21.532493 21801 raft_consensus.cc:2454] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ee2accedc6d49418052fb60d9e88daa in term 0.
I20260619 02:45:21.532632 22016 leader_election.cc:302] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0050f242fb53482db19392387217e533, 1ee2accedc6d49418052fb60d9e88daa; no voters:
I20260619 02:45:21.532732 22217 raft_consensus.cc:2781] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:21.532783 22217 raft_consensus.cc:482] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:21.532801 22217 raft_consensus.cc:3037] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:21.533488 22217 raft_consensus.cc:504] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.533598 22217 leader_election.cc:288] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [CANDIDATE]: Term 1 election: Requested vote from peers 0050f242fb53482db19392387217e533 (127.19.113.65:46457), 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665)
I20260619 02:45:21.533753 21942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09ab0007b66446884373dbcebea8f35" candidate_uuid: "1ee2accedc6d49418052fb60d9e88daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28210ebc05944a74a01e05f4f42b24e6"
I20260619 02:45:21.533789 21801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09ab0007b66446884373dbcebea8f35" candidate_uuid: "1ee2accedc6d49418052fb60d9e88daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0050f242fb53482db19392387217e533"
I20260619 02:45:21.533831 21942 raft_consensus.cc:3037] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:21.533851 21801 raft_consensus.cc:3037] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:21.534433 21801 raft_consensus.cc:2454] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee2accedc6d49418052fb60d9e88daa in term 1.
I20260619 02:45:21.534451 21942 raft_consensus.cc:2454] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee2accedc6d49418052fb60d9e88daa in term 1.
I20260619 02:45:21.534588 22013 leader_election.cc:302] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee2accedc6d49418052fb60d9e88daa, 28210ebc05944a74a01e05f4f42b24e6; no voters:
I20260619 02:45:21.534677 22217 raft_consensus.cc:2781] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:21.534723 22217 raft_consensus.cc:686] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 1 LEADER]: Becoming Leader. State: Replica: 1ee2accedc6d49418052fb60d9e88daa, State: Running, Role: LEADER
I20260619 02:45:21.534768 22217 consensus_queue.cc:237] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } }
I20260619 02:45:21.535285 21644 catalog_manager.cc:5495] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67). New cstate: current_term: 1 leader_uuid: "1ee2accedc6d49418052fb60d9e88daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ee2accedc6d49418052fb60d9e88daa" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40499 } health_report { overall_health: HEALTHY } } }
I20260619 02:45:21.541424 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:21.539220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.19.113.67:40499)
0619 02:45:21.539379 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:21.539381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:21.539387 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:45:21.539410 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:21.539410 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:21.539419 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:21.539472 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:21.539590 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.540069 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.540769 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.540984 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.541036 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:21.541163 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:21.541205 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:21.541246 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:21.541308 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:45:21.541332 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260619 02:45:21.545770 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:21.543499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40499 (local address 127.19.113.67:45235)
0619 02:45:21.543612 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:21.543622 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:21.543668 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:21.543776 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:21.543776 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:21.543778 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:21.543817 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:21.543818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.544481 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.544482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:21.545076 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:21.545078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.545425 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.545426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:21.545469 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:21.545474 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:21.545621 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:21.545622 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:45:21.545655 (+ 33us) client_negotiation.cc:240] Negotiation successful
0619 02:45:21.545692 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
I20260619 02:45:21.545769 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:21.543624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:45235 (local address 127.19.113.67:40499)
0619 02:45:21.543691 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:21.543693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:21.543699 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:45:21.543716 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:21.543716 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:21.543725 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:21.543768 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:21.543884 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.544415 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.545153 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:21.545360 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:21.545412 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:21.545553 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:21.545575 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:21.545608 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:21.545660 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 02:45:21.545682 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:45:21.547627 21801 raft_consensus.cc:1261] T e09ab0007b66446884373dbcebea8f35 P 0050f242fb53482db19392387217e533 [term 1 FOLLOWER]: Refusing update from remote peer 1ee2accedc6d49418052fb60d9e88daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:21.547762 21942 raft_consensus.cc:1261] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Refusing update from remote peer 1ee2accedc6d49418052fb60d9e88daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:21.547813 22156 consensus_queue.cc:1035] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0050f242fb53482db19392387217e533" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:21.547935 22212 consensus_queue.cc:1035] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "28210ebc05944a74a01e05f4f42b24e6" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:45:21.551828 22210 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:45:23.069427 21645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:23.069537 21645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:23.070472 21645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de59d6dc757c45d4b7a7853c7c0b2c31: Sending DeleteTablet for 3 replicas of tablet e09ab0007b66446884373dbcebea8f35
I20260619 02:45:23.070731 21922 tablet_service.cc:1514] Processing DeleteTablet for tablet e09ab0007b66446884373dbcebea8f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:23 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:51938
I20260619 02:45:23.070750 22062 tablet_service.cc:1514] Processing DeleteTablet for tablet e09ab0007b66446884373dbcebea8f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:23 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:37462
I20260619 02:45:23.070931 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 21717
I20260619 02:45:23.070998 21782 tablet_service.cc:1514] Processing DeleteTablet for tablet e09ab0007b66446884373dbcebea8f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:23 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:40640
I20260619 02:45:23.070993 22286 tablet_replica.cc:330] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: stopping tablet replica
I20260619 02:45:23.071548 22286 raft_consensus.cc:2229] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:23.071648 22286 raft_consensus.cc:2258] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:23.071866 22286 ts_tablet_manager.cc:1905] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:23.073319 22286 ts_tablet_manager.cc:1918] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:45:23.073370 22286 log.cc:1196] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/e09ab0007b66446884373dbcebea8f35
I20260619 02:45:23.073581 22286 ts_tablet_manager.cc:1939] T e09ab0007b66446884373dbcebea8f35 P 1ee2accedc6d49418052fb60d9e88daa: Deleting consensus metadata
I20260619 02:45:23.073891 21629 catalog_manager.cc:4841] TS 1ee2accedc6d49418052fb60d9e88daa (127.19.113.67:40499): tablet e09ab0007b66446884373dbcebea8f35 (table test-table [id=8e69121ea69242c2beb95d158f40867f]) successfully deleted
I20260619 02:45:23.075022 22285 tablet_replica.cc:330] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: stopping tablet replica
I20260619 02:45:23.075122 22285 raft_consensus.cc:2229] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:23.075176 22285 raft_consensus.cc:2258] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:23.075393 22285 ts_tablet_manager.cc:1905] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:23.076604 22285 ts_tablet_manager.cc:1918] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:45:23.076653 22285 log.cc:1196] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/e09ab0007b66446884373dbcebea8f35
I20260619 02:45:23.076857 22285 ts_tablet_manager.cc:1939] T e09ab0007b66446884373dbcebea8f35 P 28210ebc05944a74a01e05f4f42b24e6: Deleting consensus metadata
W20260619 02:45:23.077124 21632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:46457 (error 108)
I20260619 02:45:23.077200 21629 catalog_manager.cc:4841] TS 28210ebc05944a74a01e05f4f42b24e6 (127.19.113.66:40665): tablet e09ab0007b66446884373dbcebea8f35 (table test-table [id=8e69121ea69242c2beb95d158f40867f]) successfully deleted
I20260619 02:45:23.077394 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 21857
I20260619 02:45:23.077987 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:23.077521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46457 (local address 127.0.0.1:40666)
0619 02:45:23.077900 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:23.077949 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":68,"threads_started":1}
W20260619 02:45:23.078435 21632 catalog_manager.cc:4584] TS 0050f242fb53482db19392387217e533 (127.19.113.65:46457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e09ab0007b66446884373dbcebea8f35: Network error: Client connection negotiation failed: client connection to 127.19.113.65:46457: connect: Connection refused (error 111)
I20260619 02:45:23.082504 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 21997
I20260619 02:45:23.087422 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 21613
2026-06-19T02:45:23Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5444 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:23 dist-test-slave-zqpb krb5kdc[22291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:23 dist-test-slave-zqpb krb5kdc[22291](info): set up 2 sockets
Jun 19 02:45:23 dist-test-slave-zqpb krb5kdc[22291](info): commencing operation
krb5kdc: starting...
W20260619 02:45:24.042460 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:24.161454 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:33467
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:33467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:24.243546 22304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:24.243749 22304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:24.243772 22304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:24.244937 22304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:24.244988 22304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:24.245002 22304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:24.245012 22304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:24.245025 22304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:24.246615 22304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:33467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:33467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:24.246856 22304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:24.247107 22304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:24.251801 22312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:24.251821 22310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:24.251904 22309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:24.252117 22304 server_base.cc:1034] running on GCE node
I20260619 02:45:24.252374 22304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:24.252655 22304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:24.253791 22304 hybrid_clock.cc:648] HybridClock initialized: now 1781837124253778 us; error 29 us; skew 500 ppm
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:24.256423 22304 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:24.256819 22304 webserver.cc:460] Webserver started at http://127.19.113.126:39257/ using document root <none> and password file <none>
I20260619 02:45:24.257109 22304 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:24.257158 22304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:24.257279 22304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:24.258435 22304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "ac1f4cc800074d04b68b511ed8ac72a9"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "fc2081dd7b40fe2358f445447cd4d8c8"
server_key_iv: "9f4aa9be1b5154b85a5237464cadd57c"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.258785 22304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "ac1f4cc800074d04b68b511ed8ac72a9"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "fc2081dd7b40fe2358f445447cd4d8c8"
server_key_iv: "9f4aa9be1b5154b85a5237464cadd57c"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.260114 22304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:24.260793 22319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.260938 22304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.261062 22304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "ac1f4cc800074d04b68b511ed8ac72a9"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "fc2081dd7b40fe2358f445447cd4d8c8"
server_key_iv: "9f4aa9be1b5154b85a5237464cadd57c"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.261133 22304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:24.272831 22304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:24.274967 22304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:24.275089 22304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:24.278730 22304 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:33467
I20260619 02:45:24.278762 22371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:33467 every 8 connection(s)
I20260619 02:45:24.279103 22304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:24.279937 22372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:24.282218 22372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9: Bootstrap starting.
I20260619 02:45:24.282781 22372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:24.283072 22372 log.cc:824] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:24.283793 22372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9: No bootstrap required, opened a new log
I20260619 02:45:24.285399 22372 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1f4cc800074d04b68b511ed8ac72a9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33467 } }
I20260619 02:45:24.285547 22372 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:24.285578 22372 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1f4cc800074d04b68b511ed8ac72a9, State: Initialized, Role: FOLLOWER
I20260619 02:45:24.285715 22372 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1f4cc800074d04b68b511ed8ac72a9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33467 } }
I20260619 02:45:24.285792 22372 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:24.285825 22372 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:24.285872 22372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:24.286245 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 22304
I20260619 02:45:24.286338 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:24.286537 19909 external_mini_cluster.cc:1428] Setting key d60aabf7516ad40972de6f6e56fef2e2
I20260619 02:45:24.286538 22372 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1f4cc800074d04b68b511ed8ac72a9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33467 } }
I20260619 02:45:24.286635 22372 leader_election.cc:302] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac1f4cc800074d04b68b511ed8ac72a9; no voters:
I20260619 02:45:24.286818 22372 leader_election.cc:288] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:24.286962 22375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:24.287101 22375 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [term 1 LEADER]: Becoming Leader. State: Replica: ac1f4cc800074d04b68b511ed8ac72a9, State: Running, Role: LEADER
I20260619 02:45:24.287137 22372 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:24.287285 22375 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1f4cc800074d04b68b511ed8ac72a9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33467 } }
I20260619 02:45:24.287655 22375 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac1f4cc800074d04b68b511ed8ac72a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1f4cc800074d04b68b511ed8ac72a9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33467 } } }
I20260619 02:45:24.287914 22375 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:24.287832 22376 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac1f4cc800074d04b68b511ed8ac72a9. Latest consensus state: current_term: 1 leader_uuid: "ac1f4cc800074d04b68b511ed8ac72a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1f4cc800074d04b68b511ed8ac72a9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33467 } } }
I20260619 02:45:24.288388 22376 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:24.288558 22386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:24.289047 22386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:24.291196 22386 catalog_manager.cc:1347] Generated new cluster ID: 706326a5d003434296221b05c5dd82f0
I20260619 02:45:24.291254 22386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:24.298652 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:24.287122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.19.113.126:33467)
0619 02:45:24.287456 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:24.287466 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:24.287500 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:45:24.287619 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:24.287626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:24.287698 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:24.288584 (+ 886us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:24.288704 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.290014 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.290823 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.291456 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.291558 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:24.294299 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:24.294304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:24.294304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:24.294317 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:24.296853 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:24.297289 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:24.297293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:24.297293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:24.297416 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:24.297598 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:24.297598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:24.297599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:24.297913 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:24.298073 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:24.298143 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:45:24.298397 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":73,"threads_started":1}
I20260619 02:45:24.306705 22386 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:24.307464 22386 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:24.320513 22386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac1f4cc800074d04b68b511ed8ac72a9: Generated new TSK 0
I20260619 02:45:24.320730 22386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:24.353689 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33467
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:24.434899 22398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:24.435134 22398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:24.435166 22398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:24.436374 22398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:24.436424 22398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:24.436460 22398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:24.437947 22398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:24.438194 22398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:24.438417 22398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:24.443372 22404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:24.443419 22406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:24.443544 22403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:24.443615 22398 server_base.cc:1034] running on GCE node
I20260619 02:45:24.443840 22398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:24.444080 22398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:24.445246 22398 hybrid_clock.cc:648] HybridClock initialized: now 1781837124445229 us; error 28 us; skew 500 ppm
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:24.447767 22398 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:24.448222 22398 webserver.cc:460] Webserver started at http://127.19.113.65:44321/ using document root <none> and password file <none>
I20260619 02:45:24.448441 22398 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:24.448486 22398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:24.448614 22398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:24.449681 22398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "265a27962ff24b51841722f8b13559ef"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "0b1ac0ff2d8ac8b65867d67f0d82b553"
server_key_iv: "1ac3024f48e02cfa36a262e1ce8ab153"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.450011 22398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "265a27962ff24b51841722f8b13559ef"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "0b1ac0ff2d8ac8b65867d67f0d82b553"
server_key_iv: "1ac3024f48e02cfa36a262e1ce8ab153"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.451393 22398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:24.452384 22413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.452600 22398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:24.452700 22398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "265a27962ff24b51841722f8b13559ef"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "0b1ac0ff2d8ac8b65867d67f0d82b553"
server_key_iv: "1ac3024f48e02cfa36a262e1ce8ab153"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.452767 22398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:24.476022 22398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:24.478019 22398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:24.478133 22398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:24.478323 22398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:24.478621 22398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:24.478653 22398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.478675 22398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:24.478698 22398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.484520 22398 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:43275
I20260619 02:45:24.484580 22526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:43275 every 8 connection(s)
I20260619 02:45:24.484903 22398 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:24.489504 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 22398
I20260619 02:45:24.489607 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:24.489781 19909 external_mini_cluster.cc:1428] Setting key 2130ead507a0e29c724dfc5527a89f79
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:24.495175 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:24.485675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:56233 (local address 127.19.113.126:33467)
0619 02:45:24.485751 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:24.485753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:24.485970 (+ 217us) server_negotiation.cc:407] Connection header received
0619 02:45:24.486472 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:24.486473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:24.486485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:24.486544 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:24.487549 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.488426 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.489584 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.489890 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.489966 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:24.492431 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:24.492433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:24.492433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:24.492443 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:24.493739 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:24.494206 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:24.494207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:24.494207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:24.494262 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:24.494493 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:24.494494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:24.494494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:24.494627 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:24.494681 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:24.494988 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 02:45:24.495080 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:45:24.495380 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:24.485633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33467 (local address 127.19.113.65:56233)
0619 02:45:24.485798 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:24.485839 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:24.486370 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:24.486607 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:24.486610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:24.487228 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:24.487452 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:24.487458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.488526 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.488527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:24.489482 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:24.489485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.489975 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.489976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:24.490106 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:24.490661 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:24.490670 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:24.492327 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:24.493825 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:24.493833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:24.493836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:24.494137 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:24.494316 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:24.494317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:24.494317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:24.494436 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:24.494683 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:24.494691 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:24.494835 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:45:24.494966 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 02:45:24.495147 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260619 02:45:24.495879 22527 heartbeater.cc:347] Connected to a master server at 127.19.113.126:33467
I20260619 02:45:24.496003 22527 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:24.496212 22527 heartbeater.cc:511] Master 127.19.113.126:33467 requested a full tablet report, sending...
I20260619 02:45:24.496811 22336 ts_manager.cc:194] Registered new tserver with Master: 265a27962ff24b51841722f8b13559ef (127.19.113.65:43275)
I20260619 02:45:24.497742 22336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:56233
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:24.515341 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33467
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:24.595481 22532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:24.595705 22532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:24.595728 22532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:24.596884 22532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:24.596921 22532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:24.596946 22532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:24.598347 22532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:24.598546 22532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:24.598724 22532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:24.603717 22540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:24.603935 22532 server_base.cc:1034] running on GCE node
W20260619 02:45:24.603724 22538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:24.603756 22537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:24.604238 22532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:24.604480 22532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:24.605616 22532 hybrid_clock.cc:648] HybridClock initialized: now 1781837124605605 us; error 27 us; skew 500 ppm
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:24.608065 22532 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:24.608482 22532 webserver.cc:460] Webserver started at http://127.19.113.66:46081/ using document root <none> and password file <none>
I20260619 02:45:24.608687 22532 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:24.608734 22532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:24.608858 22532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:24.609968 22532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "e1d1da14585640228ec286c9935bb8a8"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "eec833b23871efc9ada9dde475886260"
server_key_iv: "7ace5dfacda4589e019fde5a761e4b50"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.610296 22532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "e1d1da14585640228ec286c9935bb8a8"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "eec833b23871efc9ada9dde475886260"
server_key_iv: "7ace5dfacda4589e019fde5a761e4b50"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.611677 22532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:24.612468 22547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.612635 22532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:24.612718 22532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "e1d1da14585640228ec286c9935bb8a8"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "eec833b23871efc9ada9dde475886260"
server_key_iv: "7ace5dfacda4589e019fde5a761e4b50"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.612782 22532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:24.623777 22532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:24.626435 22532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:24.626588 22532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:24.626827 22532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:24.627197 22532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:24.627231 22532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.627259 22532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:24.627276 22532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.632943 22532 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:46487
I20260619 02:45:24.632995 22660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:46487 every 8 connection(s)
I20260619 02:45:24.633345 22532 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:24.640077 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 22532
I20260619 02:45:24.640172 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:24.640363 19909 external_mini_cluster.cc:1428] Setting key c4e21998125bc5e38783f7ce5fa2484a
I20260619 02:45:24.642833 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:24.634161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:43231 (local address 127.19.113.126:33467)
0619 02:45:24.634226 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:24.634229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:24.634367 (+ 138us) server_negotiation.cc:407] Connection header received
0619 02:45:24.634902 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:24.634903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:24.634915 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:24.634967 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:24.635925 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.636427 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.637410 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.637602 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.637657 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:24.639937 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:24.639938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:24.639938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:24.639946 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:24.641470 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:24.641875 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:24.641876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:24.641876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:24.641933 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:24.642207 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:24.642208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:24.642208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:24.642329 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:24.642385 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:24.642652 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:45:24.642738 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:45:24.643041 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:24.634040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33467 (local address 127.19.113.66:43231)
0619 02:45:24.634227 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:24.634277 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:24.634785 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:24.635038 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:24.635042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:24.635630 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:24.635826 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:24.635831 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.636517 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.636518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:24.637320 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:24.637322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.637669 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.637669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:24.637790 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:24.638277 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:24.638286 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:24.639820 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:24.641573 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:24.641580 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:24.641583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:24.641808 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:24.641987 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:24.641988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:24.641988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:24.642145 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:24.642384 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:24.642391 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:24.642533 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:45:24.642623 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:45:24.642783 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260619 02:45:24.643507 22661 heartbeater.cc:347] Connected to a master server at 127.19.113.126:33467
I20260619 02:45:24.643595 22661 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:24.643765 22661 heartbeater.cc:511] Master 127.19.113.126:33467 requested a full tablet report, sending...
I20260619 02:45:24.644137 22336 ts_manager.cc:194] Registered new tserver with Master: e1d1da14585640228ec286c9935bb8a8 (127.19.113.66:46487)
I20260619 02:45:24.644523 22336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:43231
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:24.663218 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33467
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:24.744956 22666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:24.745189 22666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:24.745222 22666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:24.746436 22666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:24.746481 22666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:24.746517 22666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:24.748028 22666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:24.748275 22666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:24.748486 22666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:24.753326 22672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:24.753366 22671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:24.753456 22666 server_base.cc:1034] running on GCE node
W20260619 02:45:24.753525 22674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:24.753830 22666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:24.754083 22666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:24.755249 22666 hybrid_clock.cc:648] HybridClock initialized: now 1781837124755232 us; error 30 us; skew 500 ppm
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:24.757792 22666 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:24.758230 22666 webserver.cc:460] Webserver started at http://127.19.113.67:40165/ using document root <none> and password file <none>
I20260619 02:45:24.758430 22666 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:24.758473 22666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:24.758610 22666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:24.759769 22666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "b04b1ff87a0e48a4bb785a8c8710f74c"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "d29a7bf9885d1037a56548584096183e"
server_key_iv: "3b8964923992cd9074931d3dcfd15cd7"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.760102 22666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "b04b1ff87a0e48a4bb785a8c8710f74c"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "d29a7bf9885d1037a56548584096183e"
server_key_iv: "3b8964923992cd9074931d3dcfd15cd7"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.761475 22666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:45:24.762264 22681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.762434 22666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:24.762552 22666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "b04b1ff87a0e48a4bb785a8c8710f74c"
format_stamp: "Formatted at 2026-06-19 02:45:24 on dist-test-slave-zqpb"
server_key: "d29a7bf9885d1037a56548584096183e"
server_key_iv: "3b8964923992cd9074931d3dcfd15cd7"
server_key_version: "encryptionkey@0"
I20260619 02:45:24.762619 22666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:24.784694 22666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:24.786840 22666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:24.787076 22666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:24.787309 22666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:24.787640 22666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:24.787673 22666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.787694 22666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:24.787715 22666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:24.793462 22666 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:35603
I20260619 02:45:24.793519 22794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:35603 every 8 connection(s)
I20260619 02:45:24.793825 22666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:24.798079 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 22666
I20260619 02:45:24.798192 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:24.798350 19909 external_mini_cluster.cc:1428] Setting key f8b051d3a2773a1d8f4f62726abc3214
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837124, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:24.803053 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:24.794539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:56387 (local address 127.19.113.126:33467)
0619 02:45:24.794585 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:24.794587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:24.794804 (+ 217us) server_negotiation.cc:407] Connection header received
0619 02:45:24.795381 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:24.795383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:24.795396 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:24.795449 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:24.796399 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.796918 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.797891 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.798082 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.798143 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:24.800435 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:24.800437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:24.800438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:24.800446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:24.801678 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:24.802091 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:24.802092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:24.802092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:24.802145 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:24.802373 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:24.802374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:24.802374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:24.802498 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:24.802556 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:24.802838 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 02:45:24.802958 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:24.803345 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:24.794504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33467 (local address 127.19.113.67:56387)
0619 02:45:24.794687 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:24.794723 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:24.795274 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:24.795525 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:24.795529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:24.796106 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:24.796305 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:24.796310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.797013 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.797014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:24.797806 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:24.797808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:24.798156 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:24.798157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:24.798300 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:24.798848 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:24.798856 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:24.800348 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:24.801767 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:24.801775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:24.801778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:24.802016 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:24.802200 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:24.802201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:24.802201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:24.802313 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:24.802555 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:24.802562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:24.802717 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 02:45:24.802807 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:45:24.803115 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260619 02:45:24.803965 22795 heartbeater.cc:347] Connected to a master server at 127.19.113.126:33467
I20260619 02:45:24.804100 22795 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:24.804258 22795 heartbeater.cc:511] Master 127.19.113.126:33467 requested a full tablet report, sending...
I20260619 02:45:24.804644 22336 ts_manager.cc:194] Registered new tserver with Master: b04b1ff87a0e48a4bb785a8c8710f74c (127.19.113.67:35603)
I20260619 02:45:24.805042 22336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:56387
I20260619 02:45:24.810422 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:45:24.821447 22379 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:24.820619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.19.113.126:33467)
0619 02:45:24.820667 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:24.820669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:24.820731 (+ 62us) server_negotiation.cc:407] Connection header received
0619 02:45:24.820784 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:24.820785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:24.820794 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:24.820841 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:24.821372 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38662: BlockingRecv error: recv got EOF from 127.0.0.1:38662 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:24.822463 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 22398
I20260619 02:45:24.826400 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 22532
I20260619 02:45:24.830765 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 22666
I20260619 02:45:24.835295 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 22304
2026-06-19T02:45:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1746 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22807](info): set up 2 sockets
Jun 19 02:45:24 dist-test-slave-zqpb krb5kdc[22807](info): commencing operation
krb5kdc: starting...
W20260619 02:45:25.794576 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:25 dist-test-slave-zqpb krb5kdc[22807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:25.911669 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:35585
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:35585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:25.993989 22820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:25.994216 22820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:25.994238 22820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:25.995458 22820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:25.995504 22820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:25.995520 22820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:25.995532 22820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:25.995543 22820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:25.997067 22820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:35585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:35585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:25.997303 22820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:25.997516 22820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:26.002324 22825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:26.002317 22826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:26.002457 22820 server_base.cc:1034] running on GCE node
W20260619 02:45:26.002321 22828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:26.002781 22820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:26.003104 22820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:26.004279 22820 hybrid_clock.cc:648] HybridClock initialized: now 1781837126004260 us; error 34 us; skew 500 ppm
Jun 19 02:45:25 dist-test-slave-zqpb krb5kdc[22807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837125, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:26.006821 22820 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:26.007274 22820 webserver.cc:460] Webserver started at http://127.19.113.126:35765/ using document root <none> and password file <none>
I20260619 02:45:26.007516 22820 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:26.007568 22820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:26.007689 22820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:26.008759 22820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9"
format_stamp: "Formatted at 2026-06-19 02:45:26 on dist-test-slave-zqpb"
server_key: "1d7833153d9466159a46dc8f341145fa"
server_key_iv: "1e8f63e68ae9fc2cfa1d9d50b38fad9e"
server_key_version: "encryptionkey@0"
I20260619 02:45:26.009099 22820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9"
format_stamp: "Formatted at 2026-06-19 02:45:26 on dist-test-slave-zqpb"
server_key: "1d7833153d9466159a46dc8f341145fa"
server_key_iv: "1e8f63e68ae9fc2cfa1d9d50b38fad9e"
server_key_version: "encryptionkey@0"
I20260619 02:45:26.010639 22820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:45:26.011443 22835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:26.011598 22820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:26.011675 22820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9"
format_stamp: "Formatted at 2026-06-19 02:45:26 on dist-test-slave-zqpb"
server_key: "1d7833153d9466159a46dc8f341145fa"
server_key_iv: "1e8f63e68ae9fc2cfa1d9d50b38fad9e"
server_key_version: "encryptionkey@0"
I20260619 02:45:26.011742 22820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:26.064270 22820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:26.066599 22820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:26.066825 22820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:26.070585 22820 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:35585
I20260619 02:45:26.070618 22887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:35585 every 8 connection(s)
I20260619 02:45:26.071092 22820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:26.071760 22888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:26.074106 22888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9: Bootstrap starting.
I20260619 02:45:26.074698 22888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:26.075013 22888 log.cc:824] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:26.075764 22888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9: No bootstrap required, opened a new log
I20260619 02:45:26.077428 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 22820
I20260619 02:45:26.077519 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:26.077670 19909 external_mini_cluster.cc:1428] Setting key 3752193f17be4c3fb06cf6a51e3b6fd0
I20260619 02:45:26.077812 22888 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35585 } }
I20260619 02:45:26.078002 22888 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:26.078042 22888 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfaa97acae8f4a64aaa4dc240d4a0fb9, State: Initialized, Role: FOLLOWER
I20260619 02:45:26.078204 22888 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35585 } }
I20260619 02:45:26.078293 22888 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:26.078323 22888 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:26.078392 22888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:26.079131 22888 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35585 } }
I20260619 02:45:26.079221 22888 leader_election.cc:302] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfaa97acae8f4a64aaa4dc240d4a0fb9; no voters:
I20260619 02:45:26.079377 22888 leader_election.cc:288] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:26.079428 22893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:26.079608 22893 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [term 1 LEADER]: Becoming Leader. State: Replica: bfaa97acae8f4a64aaa4dc240d4a0fb9, State: Running, Role: LEADER
I20260619 02:45:26.079735 22888 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:26.079746 22893 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35585 } }
I20260619 02:45:26.080159 22894 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35585 } } }
I20260619 02:45:26.080241 22894 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:26.080443 22895 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfaa97acae8f4a64aaa4dc240d4a0fb9. Latest consensus state: current_term: 1 leader_uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa97acae8f4a64aaa4dc240d4a0fb9" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 35585 } } }
I20260619 02:45:26.080758 22895 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:26.080921 22899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:26.081476 22899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:26.083300 22899 catalog_manager.cc:1347] Generated new cluster ID: 297b33b02e0b440d98344db99d332dc4
I20260619 02:45:26.083351 22899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:26 dist-test-slave-zqpb krb5kdc[22807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:26.087453 22899 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:26.088004 22899 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:26.089521 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:26.078251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58272 (local address 127.19.113.126:35585)
0619 02:45:26.079019 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:26.079028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:26.079054 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:45:26.079149 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:26.079154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:26.079202 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:26.079378 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:26.080153 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:26.081470 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:26.082238 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:26.082853 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:26.083002 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:26.085759 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:26.085763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:26.085763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:26.085776 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:26.087740 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:26.088155 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:26.088159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:26.088159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:26.088274 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:26.088615 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:26.088616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:26.088616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:26.088925 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:26.089078 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:26.089155 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:45:26.089304 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":527,"threads_started":1}
I20260619 02:45:26.090615 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:26.094220 22899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfaa97acae8f4a64aaa4dc240d4a0fb9: Generated new TSK 0
I20260619 02:45:26.094393 22899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:26.105679 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:26.102303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.19.113.126:35585)
0619 02:45:26.102358 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:26.102361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:26.102481 (+ 120us) server_negotiation.cc:407] Connection header received
0619 02:45:26.102508 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:26.102509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:26.102521 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:26.102563 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:26.103024 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:26.103552 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:26.104259 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:26.104477 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:26.104577 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:26.105207 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:26.105208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:26.105208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:26.105428 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:26.105444 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:26.105450 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:26.105497 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:26.105566 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:45:26.105595 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:45:26.107825 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 22820
2026-06-19T02:45:26Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1271 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:26 dist-test-slave-zqpb krb5kdc[22921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:26 dist-test-slave-zqpb krb5kdc[22921](info): set up 2 sockets
Jun 19 02:45:26 dist-test-slave-zqpb krb5kdc[22921](info): commencing operation
krb5kdc: starting...
W20260619 02:45:27.062645 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:27.179251 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45597
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:45597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:27.260231 22934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:27.260443 22934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:27.260464 22934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:27.261574 22934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:27.261611 22934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:27.261624 22934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:27.261636 22934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:27.261646 22934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:27.263109 22934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:45597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:27.263302 22934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:27.263505 22934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:27.267978 22940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:27.268052 22942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:27.268186 22934 server_base.cc:1034] running on GCE node
W20260619 02:45:27.268028 22939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:27.268476 22934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:27.268733 22934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:27.269881 22934 hybrid_clock.cc:648] HybridClock initialized: now 1781837127269865 us; error 31 us; skew 500 ppm
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:27.272343 22934 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:27.272758 22934 webserver.cc:460] Webserver started at http://127.19.113.126:35957/ using document root <none> and password file <none>
I20260619 02:45:27.272969 22934 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:27.273058 22934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:27.273180 22934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:27.274240 22934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "6f206dd3dfac449390e2c3b15faf3198"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "d6a834890653a1595d7e0c19de74735b"
server_key_iv: "6b3b12699372b32afb716192ef996b6c"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.274550 22934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "6f206dd3dfac449390e2c3b15faf3198"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "d6a834890653a1595d7e0c19de74735b"
server_key_iv: "6b3b12699372b32afb716192ef996b6c"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.275799 22934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:45:27.276600 22949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.276762 22934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:27.276839 22934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "6f206dd3dfac449390e2c3b15faf3198"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "d6a834890653a1595d7e0c19de74735b"
server_key_iv: "6b3b12699372b32afb716192ef996b6c"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.276911 22934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:27.288434 22934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:27.290232 22934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:27.290355 22934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:27.294027 22934 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:45597
I20260619 02:45:27.294072 23001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:45597 every 8 connection(s)
I20260619 02:45:27.294384 22934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:27.295419 23002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:27.297680 23002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198: Bootstrap starting.
I20260619 02:45:27.298235 23002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:27.298492 23002 log.cc:824] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:27.299283 23002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198: No bootstrap required, opened a new log
I20260619 02:45:27.300868 23002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f206dd3dfac449390e2c3b15faf3198" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45597 } }
I20260619 02:45:27.301019 23002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:27.301051 23002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f206dd3dfac449390e2c3b15faf3198, State: Initialized, Role: FOLLOWER
I20260619 02:45:27.301182 23002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f206dd3dfac449390e2c3b15faf3198" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45597 } }
I20260619 02:45:27.301241 23002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:27.301278 23002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:27.301326 23002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:27.302202 23002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f206dd3dfac449390e2c3b15faf3198" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45597 } }
I20260619 02:45:27.302299 23002 leader_election.cc:302] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f206dd3dfac449390e2c3b15faf3198; no voters:
I20260619 02:45:27.302459 23002 leader_election.cc:288] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:27.302501 23005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:27.302647 23005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [term 1 LEADER]: Becoming Leader. State: Replica: 6f206dd3dfac449390e2c3b15faf3198, State: Running, Role: LEADER
I20260619 02:45:27.302713 23002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:27.302759 23005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f206dd3dfac449390e2c3b15faf3198" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45597 } }
I20260619 02:45:27.303145 23007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f206dd3dfac449390e2c3b15faf3198. Latest consensus state: current_term: 1 leader_uuid: "6f206dd3dfac449390e2c3b15faf3198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f206dd3dfac449390e2c3b15faf3198" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45597 } } }
I20260619 02:45:27.303251 23007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:27.303144 23006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f206dd3dfac449390e2c3b15faf3198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f206dd3dfac449390e2c3b15faf3198" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45597 } } }
I20260619 02:45:27.303339 23006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:27.303486 23014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:27.304013 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 22934
I20260619 02:45:27.304055 23014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:27.304118 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:27.304291 19909 external_mini_cluster.cc:1428] Setting key fc821ea32c798b7377542633f45e5971
I20260619 02:45:27.305920 23014 catalog_manager.cc:1347] Generated new cluster ID: d5afe2533a3e41799daae3bc0354a804
I20260619 02:45:27.305972 23014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:27.315533 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:27.304776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.19.113.126:45597)
0619 02:45:27.305006 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:27.305015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:27.305043 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:45:27.305131 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:27.305136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:27.305178 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:27.305287 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:27.306047 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.307575 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.308659 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.309245 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.309334 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:27.312026 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:27.312030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:27.312031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:27.312043 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:27.313945 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.314305 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.314309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.314309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.314415 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.314578 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.314578 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.314579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.314908 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:27.315084 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:27.315152 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:45:27.315293 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
I20260619 02:45:27.329756 23014 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:27.330307 23014 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:27.335228 23014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f206dd3dfac449390e2c3b15faf3198: Generated new TSK 0
I20260619 02:45:27.335402 23014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:27.361466 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45597
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:27.441509 23028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:27.441712 23028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:27.441735 23028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:27.442849 23028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:27.442922 23028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:27.442950 23028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:27.444371 23028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:27.444551 23028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:27.444785 23028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:27.449601 23036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:27.449695 23028 server_base.cc:1034] running on GCE node
W20260619 02:45:27.449607 23034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:27.449612 23033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:27.450059 23028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:27.450285 23028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:27.451431 23028 hybrid_clock.cc:648] HybridClock initialized: now 1781837127451413 us; error 28 us; skew 500 ppm
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:27.453748 23028 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:27.454136 23028 webserver.cc:460] Webserver started at http://127.19.113.65:40331/ using document root <none> and password file <none>
I20260619 02:45:27.454344 23028 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:27.454391 23028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:27.454519 23028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:27.455575 23028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "84b2983780be4364ac6bd45ad84e6146"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "f2bb0d7597ceee831696db81f5c16185"
server_key_iv: "f83336f1e1df0229977eab01e69fc952"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.455904 23028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "84b2983780be4364ac6bd45ad84e6146"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "f2bb0d7597ceee831696db81f5c16185"
server_key_iv: "f83336f1e1df0229977eab01e69fc952"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.457108 23028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:27.457937 23043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.458133 23028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:27.458222 23028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "84b2983780be4364ac6bd45ad84e6146"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "f2bb0d7597ceee831696db81f5c16185"
server_key_iv: "f83336f1e1df0229977eab01e69fc952"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.458292 23028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:27.465619 23028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:27.467490 23028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:27.467617 23028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:27.467808 23028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:27.468080 23028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:27.468112 23028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.468139 23028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:27.468163 23028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.473490 23028 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:36069
I20260619 02:45:27.473538 23156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:36069 every 8 connection(s)
I20260619 02:45:27.473810 23028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:27.475919 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 23028
I20260619 02:45:27.476011 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:27.476181 19909 external_mini_cluster.cc:1428] Setting key d891275fbde4c4a93cbcf1abdfeb4baf
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:27.483085 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:27.474481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:36065 (local address 127.19.113.126:45597)
0619 02:45:27.474526 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:27.474529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:27.474733 (+ 204us) server_negotiation.cc:407] Connection header received
0619 02:45:27.475269 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:27.475271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:27.475286 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:27.475332 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:27.476300 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.476901 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.477927 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.478143 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.478198 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:27.480313 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:27.480314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:27.480315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:27.480323 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:27.481787 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.482140 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.482142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.482142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.482204 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.482404 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.482405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.482405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.482558 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:27.482630 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:27.482843 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 02:45:27.482976 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:27.483125 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:27.474447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45597 (local address 127.19.113.65:36065)
0619 02:45:27.474597 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:27.474635 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:27.475168 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:27.475411 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:27.475415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:27.476006 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:27.476222 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:27.476227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.476990 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.476991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:27.477840 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:27.477842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.478214 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.478215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:27.478350 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:27.478839 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:27.478848 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:27.480218 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:27.481862 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:27.481868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:27.481870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:27.482074 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:27.482267 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:27.482268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:27.482268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:27.482354 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:27.482617 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:27.482620 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:27.482728 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 02:45:27.482803 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:45:27.482952 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260619 02:45:27.483588 23157 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45597
I20260619 02:45:27.483685 23157 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:27.483853 23157 heartbeater.cc:511] Master 127.19.113.126:45597 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
I20260619 02:45:27.484390 22966 ts_manager.cc:194] Registered new tserver with Master: 84b2983780be4364ac6bd45ad84e6146 (127.19.113.65:36069)
I20260619 02:45:27.485020 22966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:36065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:27.499640 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45597
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:27.578369 23162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:27.578578 23162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:27.578599 23162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:27.579752 23162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:27.579792 23162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:27.579818 23162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:27.581259 23162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:27.581461 23162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:27.581653 23162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:27.586346 23168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:27.586323 23170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:27.586326 23167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:27.586695 23162 server_base.cc:1034] running on GCE node
I20260619 02:45:27.586922 23162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:27.587127 23162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:27.588274 23162 hybrid_clock.cc:648] HybridClock initialized: now 1781837127588255 us; error 28 us; skew 500 ppm
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:27.590588 23162 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:27.591077 23162 webserver.cc:460] Webserver started at http://127.19.113.66:39591/ using document root <none> and password file <none>
I20260619 02:45:27.591288 23162 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:27.591336 23162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:27.591457 23162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:27.592459 23162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "afbbb516b5584a44b4b4ddafa92096f7"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "633a21ab93bda11daf2c48aa67ff4526"
server_key_iv: "4ef43ac89f66ab0c1d7385ccc667cfc7"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.592794 23162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "afbbb516b5584a44b4b4ddafa92096f7"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "633a21ab93bda11daf2c48aa67ff4526"
server_key_iv: "4ef43ac89f66ab0c1d7385ccc667cfc7"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.594167 23162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:27.594975 23177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.595145 23162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:27.595227 23162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "afbbb516b5584a44b4b4ddafa92096f7"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "633a21ab93bda11daf2c48aa67ff4526"
server_key_iv: "4ef43ac89f66ab0c1d7385ccc667cfc7"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.595294 23162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:27.623786 23162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:27.625753 23162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:27.625892 23162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:27.626111 23162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:27.626434 23162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:27.626470 23162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.626499 23162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:27.626526 23162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.632233 23162 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:36825
I20260619 02:45:27.632279 23290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:36825 every 8 connection(s)
I20260619 02:45:27.632627 23162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:27.634208 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 23162
I20260619 02:45:27.634298 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:27.634460 19909 external_mini_cluster.cc:1428] Setting key 49100b81b9978b37850662804dd56f0c
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
I20260619 02:45:27.642227 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:27.633283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:48683 (local address 127.19.113.126:45597)
0619 02:45:27.633344 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:27.633346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:27.633522 (+ 176us) server_negotiation.cc:407] Connection header received
0619 02:45:27.634069 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:27.634070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:27.634083 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:27.634140 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:27.635256 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.635753 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.637090 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.637283 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.637339 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:27.639669 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:27.639671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:27.639671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:27.639679 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:27.640928 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.641334 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.641336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.641336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.641390 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.641600 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.641601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.641601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.641711 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:27.641766 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:27.642060 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 02:45:27.642139 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:45:27.642382 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:27.633247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45597 (local address 127.19.113.66:48683)
0619 02:45:27.633410 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:27.633448 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:27.633933 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:27.634212 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:27.634216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:27.634913 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:27.635175 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:27.635181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.635832 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.635833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:27.636971 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:27.636974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.637374 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.637375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:27.637521 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:27.637996 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:27.638006 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:27.639581 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:27.641036 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:27.641043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:27.641045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:27.641272 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:27.641442 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:27.641442 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:27.641443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:27.641547 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:27.641763 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:27.641769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:27.641902 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 02:45:27.642021 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:45:27.642175 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260619 02:45:27.642802 23291 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45597
I20260619 02:45:27.642912 23291 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:27.643121 23291 heartbeater.cc:511] Master 127.19.113.126:45597 requested a full tablet report, sending...
I20260619 02:45:27.643468 22966 ts_manager.cc:194] Registered new tserver with Master: afbbb516b5584a44b4b4ddafa92096f7 (127.19.113.66:36825)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
I20260619 02:45:27.643913 22966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:48683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:27.655798 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45597
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:27.734676 23296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:27.734908 23296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:27.734942 23296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:27.736110 23296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:27.736148 23296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:27.736173 23296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:27.737564 23296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:27.737748 23296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:27.737920 23296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:45:27.742733 23296 server_base.cc:1034] running on GCE node
W20260619 02:45:27.742719 23301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:27.742712 23302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:27.742719 23304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:27.743199 23296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:27.743438 23296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:27.744625 23296 hybrid_clock.cc:648] HybridClock initialized: now 1781837127744608 us; error 30 us; skew 500 ppm
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:27.747058 23296 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:27.747452 23296 webserver.cc:460] Webserver started at http://127.19.113.67:33187/ using document root <none> and password file <none>
I20260619 02:45:27.747635 23296 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:27.747673 23296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:27.747762 23296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:27.748752 23296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "9e91393448e44522938eb576d82ac06e"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "5f1135b207076d361f6b3e6098097200"
server_key_iv: "2821a11ff34fa9a060ee48eeec967e8a"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.749039 23296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "9e91393448e44522938eb576d82ac06e"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "5f1135b207076d361f6b3e6098097200"
server_key_iv: "2821a11ff34fa9a060ee48eeec967e8a"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.750257 23296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:45:27.750947 23311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.751087 23296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:27.751166 23296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "9e91393448e44522938eb576d82ac06e"
format_stamp: "Formatted at 2026-06-19 02:45:27 on dist-test-slave-zqpb"
server_key: "5f1135b207076d361f6b3e6098097200"
server_key_iv: "2821a11ff34fa9a060ee48eeec967e8a"
server_key_version: "encryptionkey@0"
I20260619 02:45:27.751220 23296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:27.766726 23296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:27.768800 23296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:27.768913 23296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:27.769085 23296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:27.769335 23296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:27.769362 23296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.769385 23296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:27.769402 23296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:27.774487 23296 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:40097
I20260619 02:45:27.774561 23424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:40097 every 8 connection(s)
I20260619 02:45:27.774819 23296 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:27.780169 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 23296
I20260619 02:45:27.780282 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:27.780485 19909 external_mini_cluster.cc:1428] Setting key 753b1f982d2d471c3541144ab223582a
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:27.783609 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:27.775478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:43749 (local address 127.19.113.126:45597)
0619 02:45:27.775555 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:27.775558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:27.775726 (+ 168us) server_negotiation.cc:407] Connection header received
0619 02:45:27.776223 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:27.776224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:27.776234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:27.776289 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:27.777175 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.777684 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.778796 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.779050 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.779112 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:27.781476 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:27.781477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:27.781477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:27.781483 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:27.782498 (+ 1015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.782819 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.782820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.782820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.782862 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.783127 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.783128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.783128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.783236 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:27.783285 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:27.783488 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 02:45:27.783546 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:45:27.783803 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:27.775453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45597 (local address 127.19.113.67:43749)
0619 02:45:27.775616 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:27.775651 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:27.776130 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:27.776341 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:27.776344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:27.776920 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:27.777101 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:27.777106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.777795 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.777796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:27.778681 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:27.778684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.779135 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.779136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:27.779253 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:27.779792 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:27.779800 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:27.781376 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:27.782560 (+ 1184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:27.782566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:27.782568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:27.782766 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:27.782937 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:27.782937 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:27.782938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:27.783069 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:27.783284 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:27.783286 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:27.783398 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 02:45:27.783473 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:45:27.783592 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260619 02:45:27.784250 23425 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45597
I20260619 02:45:27.784332 23425 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:27.784483 23425 heartbeater.cc:511] Master 127.19.113.126:45597 requested a full tablet report, sending...
I20260619 02:45:27.784785 22966 ts_manager.cc:194] Registered new tserver with Master: 9e91393448e44522938eb576d82ac06e (127.19.113.67:40097)
I20260619 02:45:27.785120 22966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:43749
I20260619 02:45:27.792126 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[22921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837127, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:27.813381 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:27.806768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39482 (local address 127.19.113.126:45597)
0619 02:45:27.806823 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:27.806826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:27.806948 (+ 122us) server_negotiation.cc:407] Connection header received
0619 02:45:27.807026 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:27.807027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:27.807038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:27.807084 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:27.807622 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.808118 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.808840 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:27.809029 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:27.809083 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:27.811153 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:27.811155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:27.811156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:27.811163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:27.812371 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.812658 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.812659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.812659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.812706 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:27.812865 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:27.812866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:27.812866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:27.813018 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:27.813104 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:27.813205 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:45:27.813280 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260619 02:45:27.813596 22966 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39482
I20260619 02:45:27.814019 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 23028
I20260619 02:45:27.817889 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 23162
I20260619 02:45:27.821784 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 23296
I20260619 02:45:27.825510 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 22934
2026-06-19T02:45:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1719 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[23437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[23437](info): set up 2 sockets
Jun 19 02:45:27 dist-test-slave-zqpb krb5kdc[23437](info): commencing operation
krb5kdc: starting...
W20260619 02:45:28.782579 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:28 dist-test-slave-zqpb krb5kdc[23437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:28.894169 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46767
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:45381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:46767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:28.972987 23450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:28.973207 23450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:28.973240 23450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:28.974395 23450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:28.974432 23450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:28.974452 23450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:28.974467 23450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:28.974484 23450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:28.976034 23450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:45381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:46767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:28.976269 23450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:28.976469 23450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:45:28.981120 23450 server_base.cc:1034] running on GCE node
W20260619 02:45:28.981201 23456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:28.981148 23458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:28.981181 23455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:28.981590 23450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:28.981866 23450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:28.983016 23450 hybrid_clock.cc:648] HybridClock initialized: now 1781837128983002 us; error 22 us; skew 500 ppm
Jun 19 02:45:28 dist-test-slave-zqpb krb5kdc[23437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837128, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:28.995159 23450 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:28.995639 23450 webserver.cc:460] Webserver started at http://127.19.113.126:40195/ using document root <none> and password file <none>
I20260619 02:45:28.995872 23450 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:28.995929 23450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:28.996057 23450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:28.997102 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "86fa64cddc9847c8a4396386cc1e728b"
format_stamp: "Formatted at 2026-06-19 02:45:28 on dist-test-slave-zqpb"
server_key: "26f83436d8dde7e226e31cd0ecade3bf"
server_key_iv: "ac219ecb5d299125606d9d3788134d00"
server_key_version: "encryptionkey@0"
I20260619 02:45:28.997442 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "86fa64cddc9847c8a4396386cc1e728b"
format_stamp: "Formatted at 2026-06-19 02:45:28 on dist-test-slave-zqpb"
server_key: "26f83436d8dde7e226e31cd0ecade3bf"
server_key_iv: "ac219ecb5d299125606d9d3788134d00"
server_key_version: "encryptionkey@0"
I20260619 02:45:28.998692 23450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:28.999492 23465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:28.999644 23450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:28.999722 23450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "86fa64cddc9847c8a4396386cc1e728b"
format_stamp: "Formatted at 2026-06-19 02:45:28 on dist-test-slave-zqpb"
server_key: "26f83436d8dde7e226e31cd0ecade3bf"
server_key_iv: "ac219ecb5d299125606d9d3788134d00"
server_key_version: "encryptionkey@0"
I20260619 02:45:28.999784 23450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:29.011900 23450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:29.014005 23450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:29.014142 23450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:29.018081 23450 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:46767
I20260619 02:45:29.018114 23517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:46767 every 8 connection(s)
I20260619 02:45:29.018604 23450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:29.019274 23518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:29.021533 23518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b: Bootstrap starting.
I20260619 02:45:29.022096 23518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:29.022359 23518 log.cc:824] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:29.023048 23518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b: No bootstrap required, opened a new log
I20260619 02:45:29.024870 23518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fa64cddc9847c8a4396386cc1e728b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46767 } }
I20260619 02:45:29.025032 23518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:29.025055 23518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86fa64cddc9847c8a4396386cc1e728b, State: Initialized, Role: FOLLOWER
I20260619 02:45:29.025197 23518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fa64cddc9847c8a4396386cc1e728b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46767 } }
I20260619 02:45:29.025274 23518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:29.025302 23518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:29.025336 23518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:29.026285 23518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fa64cddc9847c8a4396386cc1e728b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46767 } }
I20260619 02:45:29.026373 23518 leader_election.cc:302] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86fa64cddc9847c8a4396386cc1e728b; no voters:
I20260619 02:45:29.026526 23518 leader_election.cc:288] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:29.026615 23521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:29.026757 23518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:29.026768 23521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [term 1 LEADER]: Becoming Leader. State: Replica: 86fa64cddc9847c8a4396386cc1e728b, State: Running, Role: LEADER
I20260619 02:45:29.026893 23521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fa64cddc9847c8a4396386cc1e728b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46767 } }
I20260619 02:45:29.027278 23522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86fa64cddc9847c8a4396386cc1e728b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fa64cddc9847c8a4396386cc1e728b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46767 } } }
I20260619 02:45:29.027352 23522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:29.027504 23523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86fa64cddc9847c8a4396386cc1e728b. Latest consensus state: current_term: 1 leader_uuid: "86fa64cddc9847c8a4396386cc1e728b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fa64cddc9847c8a4396386cc1e728b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46767 } } }
I20260619 02:45:29.027559 23530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:29.027567 23523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:29.028139 23530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:29.028793 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 23450
I20260619 02:45:29.028893 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:29.029066 19909 external_mini_cluster.cc:1428] Setting key 0cd21e1cf2f7cdc80cc936fac687c995
I20260619 02:45:29.029647 23530 catalog_manager.cc:1347] Generated new cluster ID: 05fcd4a737fa414f8263e5747df7b20f
I20260619 02:45:29.029693 23530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:29.040308 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.029814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39994 (local address 127.19.113.126:46767)
0619 02:45:29.029975 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:29.029985 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:29.030015 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:45:29.030141 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:29.030149 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:29.030204 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:29.030346 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:29.031034 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.032484 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.033483 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.034128 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.034251 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.036951 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:29.036955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:29.036956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:29.036969 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:29.038734 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.039134 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.039137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.039137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.039231 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.039410 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.039410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.039411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.039733 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:29.039891 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:29.039959 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:45:29.040083 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260619 02:45:29.049084 23530 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:29.049811 23530 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:29.060709 23530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86fa64cddc9847c8a4396386cc1e728b: Generated new TSK 0
I20260619 02:45:29.060909 23530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:29.089300 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46767
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:45381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:29.171641 23544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:29.171849 23544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:29.171873 23544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:29.172996 23544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:29.173029 23544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:29.173053 23544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:29.174516 23544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:45381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:29.174693 23544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:29.174890 23544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:45:29.179600 23544 server_base.cc:1034] running on GCE node
W20260619 02:45:29.179560 23549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:29.179538 23550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:29.179538 23552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:29.180011 23544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:29.180258 23544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:29.181411 23544 hybrid_clock.cc:648] HybridClock initialized: now 1781837129181396 us; error 34 us; skew 500 ppm
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:29.183804 23544 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:29.184242 23544 webserver.cc:460] Webserver started at http://127.19.113.65:40443/ using document root <none> and password file <none>
I20260619 02:45:29.184437 23544 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:29.184489 23544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:29.184594 23544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:29.185599 23544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "ba9320ef746642e3a483914c303ff748"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "57efdc8da3794f3c7adc18109dc42ca0"
server_key_iv: "c26720e0f3e13c337acc5c14275dd895"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.185916 23544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "ba9320ef746642e3a483914c303ff748"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "57efdc8da3794f3c7adc18109dc42ca0"
server_key_iv: "c26720e0f3e13c337acc5c14275dd895"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.187186 23544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:45:29.188028 23559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.188202 23544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:29.188284 23544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "ba9320ef746642e3a483914c303ff748"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "57efdc8da3794f3c7adc18109dc42ca0"
server_key_iv: "c26720e0f3e13c337acc5c14275dd895"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.188342 23544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:29.206036 23544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:29.207923 23544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:29.208058 23544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:29.208271 23544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:29.208557 23544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:29.208588 23544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.208617 23544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:29.208638 23544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.214188 23544 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:34777
I20260619 02:45:29.214243 23672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:34777 every 8 connection(s)
I20260619 02:45:29.214543 23544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:29.223647 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.215260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:53023 (local address 127.19.113.126:46767)
0619 02:45:29.215320 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:29.215322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:29.215521 (+ 199us) server_negotiation.cc:407] Connection header received
0619 02:45:29.216473 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:29.216475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:29.216488 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:29.216545 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:29.217156 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.217656 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.218682 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.218965 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.219030 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.221078 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:29.221080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:29.221080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:29.221086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:29.222308 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.222667 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.222667 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.222668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.222712 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.223047 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.223048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.223049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.223163 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:29.223224 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:29.223505 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 02:45:29.223579 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:45:29.223788 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.215222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46767 (local address 127.19.113.65:53023)
0619 02:45:29.215393 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:29.215443 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:29.216377 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:29.216608 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:29.216610 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:29.216895 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:29.217089 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:29.217094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.217736 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.217738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:29.218584 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:29.218586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.219041 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.219042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:29.219150 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.219659 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:29.219667 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:29.220996 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:29.222384 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:29.222390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:29.222392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:29.222612 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:29.222781 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:29.222782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:29.222782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:29.222931 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:29.223254 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:29.223260 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:29.223382 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 02:45:29.223477 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:45:29.223604 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260619 02:45:29.223905 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 23544
I20260619 02:45:29.224015 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:29.224200 19909 external_mini_cluster.cc:1428] Setting key 7dc5f6a78953651650f6323ab7ee068a
I20260619 02:45:29.224318 23673 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46767
I20260619 02:45:29.224416 23673 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:29.224591 23673 heartbeater.cc:511] Master 127.19.113.126:46767 requested a full tablet report, sending...
I20260619 02:45:29.225062 23482 ts_manager.cc:194] Registered new tserver with Master: ba9320ef746642e3a483914c303ff748 (127.19.113.65:34777)
I20260619 02:45:29.225656 23482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:53023
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:29.245370 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46767
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:45381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:29.323956 23678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:29.324155 23678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:29.324185 23678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:29.325307 23678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:29.325346 23678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:29.325380 23678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:29.326943 23678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:45381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:29.327163 23678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:29.327373 23678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:29.332510 23684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:29.332657 23678 server_base.cc:1034] running on GCE node
W20260619 02:45:29.332510 23686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:29.332494 23683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:29.332952 23678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:29.333214 23678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:29.334365 23678 hybrid_clock.cc:648] HybridClock initialized: now 1781837129334351 us; error 29 us; skew 500 ppm
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:29.336830 23678 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:29.337265 23678 webserver.cc:460] Webserver started at http://127.19.113.66:35251/ using document root <none> and password file <none>
I20260619 02:45:29.337481 23678 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:29.337527 23678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:29.337646 23678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:29.338655 23678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "76603c862a0344dc8929c9c4d2cb33ea"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "655a4dc0c8cabee9339cc3592253c7ec"
server_key_iv: "3f458d9232a753f8769224416b62b480"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.339026 23678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "76603c862a0344dc8929c9c4d2cb33ea"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "655a4dc0c8cabee9339cc3592253c7ec"
server_key_iv: "3f458d9232a753f8769224416b62b480"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.340269 23678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:29.340907 23693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.341058 23678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:29.341136 23678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "76603c862a0344dc8929c9c4d2cb33ea"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "655a4dc0c8cabee9339cc3592253c7ec"
server_key_iv: "3f458d9232a753f8769224416b62b480"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.341197 23678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:29.358206 23678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:29.360291 23678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:29.360424 23678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:29.360643 23678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:29.360939 23678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:29.360971 23678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.361001 23678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:29.361022 23678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.366418 23678 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:34665
I20260619 02:45:29.366478 23806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:34665 every 8 connection(s)
I20260619 02:45:29.366755 23678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:29.369901 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 23678
I20260619 02:45:29.369992 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:29.370141 19909 external_mini_cluster.cc:1428] Setting key 4f7067eae2e094c319b6e9730879edc6
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:29.376155 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.367507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:40681 (local address 127.19.113.126:46767)
0619 02:45:29.367590 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:29.367593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:29.367723 (+ 130us) server_negotiation.cc:407] Connection header received
0619 02:45:29.368656 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:29.368657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:29.368668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:29.368717 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:29.369260 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.369774 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.371098 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.371364 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.371433 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.373548 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:29.373550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:29.373550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:29.373557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:29.374862 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.375297 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.375298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.375298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.375352 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.375568 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.375569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.375569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.375677 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:29.375733 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:29.375981 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 02:45:29.376065 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:45:29.376341 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.367470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46767 (local address 127.19.113.66:40681)
0619 02:45:29.367611 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:29.367647 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:29.368562 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:29.368771 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:29.368774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:29.369014 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:29.369199 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:29.369203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.369854 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.369855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:29.371021 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:29.371023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.371480 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.371481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:29.371591 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.372104 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:29.372115 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:29.373447 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:29.374998 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:29.375004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:29.375006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:29.375225 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:29.375405 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:29.375405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:29.375406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:29.375512 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:29.375728 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:29.375735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:29.375869 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 02:45:29.375961 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:45:29.376116 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260619 02:45:29.376828 23807 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46767
I20260619 02:45:29.376912 23807 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:29.377079 23807 heartbeater.cc:511] Master 127.19.113.126:46767 requested a full tablet report, sending...
I20260619 02:45:29.377449 23482 ts_manager.cc:194] Registered new tserver with Master: 76603c862a0344dc8929c9c4d2cb33ea (127.19.113.66:34665)
I20260619 02:45:29.377976 23482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:40681
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:29.394119 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46767
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:45381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:29.472608 23812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:29.472805 23812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:29.472826 23812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:29.473985 23812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:29.474022 23812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:29.474047 23812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:29.475594 23812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:45381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:29.475804 23812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:29.476091 23812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:29.480813 23817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:29.480796 23818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:29.480796 23820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:29.480854 23812 server_base.cc:1034] running on GCE node
I20260619 02:45:29.481180 23812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:29.481392 23812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:29.482548 23812 hybrid_clock.cc:648] HybridClock initialized: now 1781837129482532 us; error 29 us; skew 500 ppm
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:29.485069 23812 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:29.485486 23812 webserver.cc:460] Webserver started at http://127.19.113.67:37395/ using document root <none> and password file <none>
I20260619 02:45:29.485673 23812 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:29.485726 23812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:29.485867 23812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:29.486845 23812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "ffb9466ba3114f7f9c7672dea07d809f"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "ed255d42df05dfcee14b351ffbf7d351"
server_key_iv: "9bfb6c38139420ff4aece3662cefd94e"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.487217 23812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "ffb9466ba3114f7f9c7672dea07d809f"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "ed255d42df05dfcee14b351ffbf7d351"
server_key_iv: "9bfb6c38139420ff4aece3662cefd94e"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.488442 23812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:29.489060 23827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.489199 23812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:29.489282 23812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "ffb9466ba3114f7f9c7672dea07d809f"
format_stamp: "Formatted at 2026-06-19 02:45:29 on dist-test-slave-zqpb"
server_key: "ed255d42df05dfcee14b351ffbf7d351"
server_key_iv: "9bfb6c38139420ff4aece3662cefd94e"
server_key_version: "encryptionkey@0"
I20260619 02:45:29.489341 23812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:29.506274 23812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:29.508181 23812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:29.508292 23812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:29.508499 23812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:29.508795 23812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:29.508826 23812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.508853 23812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:29.508872 23812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:29.514099 23812 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:38139
I20260619 02:45:29.514158 23940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:38139 every 8 connection(s)
I20260619 02:45:29.514457 23812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:29.518721 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 23812
I20260619 02:45:29.518808 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:29.518986 19909 external_mini_cluster.cc:1428] Setting key c70f7768f52ff5e4cb611f35d1ddf97b
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:29.523437 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.515149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:55731 (local address 127.19.113.126:46767)
0619 02:45:29.515203 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:29.515205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:29.515417 (+ 212us) server_negotiation.cc:407] Connection header received
0619 02:45:29.516350 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:29.516352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:29.516372 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:29.516427 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:29.517046 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.517555 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.518557 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.518780 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.518862 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.521079 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:29.521080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:29.521081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:29.521088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:29.522167 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.522532 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.522533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.522534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.522586 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.522790 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.522791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.522792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.522960 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:29.523020 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:29.523269 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 02:45:29.523349 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:45:29.523597 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.515138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46767 (local address 127.19.113.67:55731)
0619 02:45:29.515278 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:29.515326 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:29.516244 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:29.516505 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:29.516508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:29.516746 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:29.516947 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:29.516952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.517642 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.517643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:29.518471 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:29.518473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.518931 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.518933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:29.519080 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.519593 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:29.519603 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:29.520993 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:29.522240 (+ 1247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:29.522247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:29.522248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:29.522466 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:29.522636 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:29.522637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:29.522637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:29.522736 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:29.523020 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:29.523026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:29.523155 (+ 129us) client_negotiation.cc:769] Sending connection context
0619 02:45:29.523239 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:45:29.523386 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260619 02:45:29.523993 23941 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46767
I20260619 02:45:29.524081 23941 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:29.524255 23941 heartbeater.cc:511] Master 127.19.113.126:46767 requested a full tablet report, sending...
I20260619 02:45:29.524600 23482 ts_manager.cc:194] Registered new tserver with Master: ffb9466ba3114f7f9c7672dea07d809f (127.19.113.67:38139)
I20260619 02:45:29.524989 23482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:55731
I20260619 02:45:29.530601 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:29.551954 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.544812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.19.113.65:34777)
0619 02:45:29.545007 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:29.545012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:29.545020 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:29.545049 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:29.545051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:29.545065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:29.545121 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:29.545656 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.546340 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.547114 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.547429 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.547499 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.549500 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:29.549506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:29.549507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:29.549513 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:29.550864 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.551201 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.551205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.551205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.551258 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.551424 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.551425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.551425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.551576 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:29.551700 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:29.551777 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:45:29.551850 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":114,"threads_started":1}
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:29.558933 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:29.553035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40004 (local address 127.19.113.126:46767)
0619 02:45:29.553087 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:29.553088 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:29.553095 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:29.553116 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:29.553117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:29.553125 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:29.553165 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:29.553538 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.554009 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.554609 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:29.554789 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:29.554833 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:29.556721 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:29.556724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:29.556724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:29.556730 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:29.557929 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.558212 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.558213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.558214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.558259 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:29.558411 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:29.558412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:29.558412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:29.558544 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:29.558626 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:29.558750 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 02:45:29.558820 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260619 02:45:29.559146 23482 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40004
I20260619 02:45:29.559448 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 23544
I20260619 02:45:29.563624 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 23678
I20260619 02:45:29.567303 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 23812
I20260619 02:45:29.570571 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 23450
2026-06-19T02:45:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1743 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23953](info): set up 2 sockets
Jun 19 02:45:29 dist-test-slave-zqpb krb5kdc[23953](info): commencing operation
krb5kdc: starting...
W20260619 02:45:30.522557 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:30 dist-test-slave-zqpb krb5kdc[23953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:30.635452 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40031
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:40031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:30.719692 23966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:30.719890 23966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:30.719920 23966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:30.721081 23966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:30.721119 23966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:30.721131 23966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:30.721143 23966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:30.721167 23966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:30.722666 23966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:40031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:30.722915 23966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:30.723153 23966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:30.727746 23974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:30.727741 23971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:30.727797 23972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:30.727835 23966 server_base.cc:1034] running on GCE node
I20260619 02:45:30.728216 23966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:30.728492 23966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:30.729637 23966 hybrid_clock.cc:648] HybridClock initialized: now 1781837130729624 us; error 27 us; skew 500 ppm
Jun 19 02:45:30 dist-test-slave-zqpb krb5kdc[23953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837130, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:30.732023 23966 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:30.732306 23966 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:30.732347 23966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:30.732439 23966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:30.733479 23966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "84ac1d5172c34bcc9e7ba9020d24343c"
format_stamp: "Formatted at 2026-06-19 02:45:30 on dist-test-slave-zqpb"
server_key: "dee65cb730aa6c05a539f537188205e1"
server_key_iv: "52b83ab938557ed3fee5c336c24c1099"
server_key_version: "encryptionkey@0"
I20260619 02:45:30.733760 23966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "84ac1d5172c34bcc9e7ba9020d24343c"
format_stamp: "Formatted at 2026-06-19 02:45:30 on dist-test-slave-zqpb"
server_key: "dee65cb730aa6c05a539f537188205e1"
server_key_iv: "52b83ab938557ed3fee5c336c24c1099"
server_key_version: "encryptionkey@0"
I20260619 02:45:30.734918 23966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:45:30.735658 23980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:30.735816 23966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:30.735898 23966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "84ac1d5172c34bcc9e7ba9020d24343c"
format_stamp: "Formatted at 2026-06-19 02:45:30 on dist-test-slave-zqpb"
server_key: "dee65cb730aa6c05a539f537188205e1"
server_key_iv: "52b83ab938557ed3fee5c336c24c1099"
server_key_version: "encryptionkey@0"
I20260619 02:45:30.735951 23966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:30.753916 23966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:30.755915 23966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:30.756038 23966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:30.759578 23966 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:40031
I20260619 02:45:30.759616 24032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:40031 every 8 connection(s)
I20260619 02:45:30.759877 23966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:30.760780 24033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:30.763037 24033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c: Bootstrap starting.
I20260619 02:45:30.763599 24033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:30.763854 24033 log.cc:824] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:30.764492 24033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c: No bootstrap required, opened a new log
I20260619 02:45:30.765995 24033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ac1d5172c34bcc9e7ba9020d24343c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40031 } }
I20260619 02:45:30.766131 24033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:30.766161 24033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84ac1d5172c34bcc9e7ba9020d24343c, State: Initialized, Role: FOLLOWER
I20260619 02:45:30.766295 24033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ac1d5172c34bcc9e7ba9020d24343c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40031 } }
I20260619 02:45:30.766369 24033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:30.766414 24033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:30.766460 24033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:30.767284 24033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ac1d5172c34bcc9e7ba9020d24343c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40031 } }
I20260619 02:45:30.767380 24033 leader_election.cc:302] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84ac1d5172c34bcc9e7ba9020d24343c; no voters:
I20260619 02:45:30.767545 24033 leader_election.cc:288] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:30.767586 24036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:30.767700 24036 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [term 1 LEADER]: Becoming Leader. State: Replica: 84ac1d5172c34bcc9e7ba9020d24343c, State: Running, Role: LEADER
I20260619 02:45:30.767796 24033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:30.767840 24036 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ac1d5172c34bcc9e7ba9020d24343c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40031 } }
I20260619 02:45:30.768231 24037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84ac1d5172c34bcc9e7ba9020d24343c. Latest consensus state: current_term: 1 leader_uuid: "84ac1d5172c34bcc9e7ba9020d24343c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ac1d5172c34bcc9e7ba9020d24343c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40031 } } }
I20260619 02:45:30.768339 24037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:30.768224 24038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84ac1d5172c34bcc9e7ba9020d24343c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ac1d5172c34bcc9e7ba9020d24343c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40031 } } }
I20260619 02:45:30.768538 24038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:30.768781 24045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:30.769233 24045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:30.769913 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 23966
I20260619 02:45:30.770021 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:30.770190 19909 external_mini_cluster.cc:1428] Setting key f4cc769d1a80462f8f13df1d32a82fcb
I20260619 02:45:30.770925 24045 catalog_manager.cc:1347] Generated new cluster ID: 0472f8e9a24541cb9b4f8503b241d9f8
I20260619 02:45:30.770975 24045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:30 dist-test-slave-zqpb krb5kdc[23953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:30.781834 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:30.770632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.19.113.126:40031)
0619 02:45:30.770758 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:30.770767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:30.771101 (+ 334us) server_negotiation.cc:407] Connection header received
0619 02:45:30.771245 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:30.771253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:30.771309 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:30.771462 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:30.772117 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:30.773572 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:30.774582 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:30.775697 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:30.775815 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:30.778430 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:30.778434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:30.778434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:30.778447 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:30.780285 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:30.780676 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:30.780678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:30.780679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:30.780775 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:30.780962 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:30.780962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:30.780963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:30.781246 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:30.781356 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:30.781467 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 02:45:30.781598 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
I20260619 02:45:30.782907 24045 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:30.783593 24045 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:30.795621 24045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c: Generated new TSK 0
I20260619 02:45:30.795837 24045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:30.820943 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40031
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:30.901724 24059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:30.901932 24059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:30.901965 24059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:30.903163 24059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:30.903203 24059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:30.903240 24059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:30.904764 24059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:30.904992 24059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:30.905203 24059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:30.909842 24064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:30.910079 24059 server_base.cc:1034] running on GCE node
W20260619 02:45:30.909817 24065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:30.909817 24067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:30.910379 24059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:30.910614 24059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:30.911746 24059 hybrid_clock.cc:648] HybridClock initialized: now 1781837130911735 us; error 26 us; skew 500 ppm
Jun 19 02:45:30 dist-test-slave-zqpb krb5kdc[23953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837130, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:30.913980 24059 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:30.914305 24059 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:30.914357 24059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:30.914539 24059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:30.915849 24059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "6a2123cd853a4390953991128360c45f"
format_stamp: "Formatted at 2026-06-19 02:45:30 on dist-test-slave-zqpb"
server_key: "5b435f609c5c91f1a0b4ea4116dc8f91"
server_key_iv: "2eb9aaf7c25205891adb8c282a780aaa"
server_key_version: "encryptionkey@0"
I20260619 02:45:30.916137 24059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "6a2123cd853a4390953991128360c45f"
format_stamp: "Formatted at 2026-06-19 02:45:30 on dist-test-slave-zqpb"
server_key: "5b435f609c5c91f1a0b4ea4116dc8f91"
server_key_iv: "2eb9aaf7c25205891adb8c282a780aaa"
server_key_version: "encryptionkey@0"
I20260619 02:45:30.917243 24059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:45:30.917866 24073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:30.917989 24059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:30.918064 24059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "6a2123cd853a4390953991128360c45f"
format_stamp: "Formatted at 2026-06-19 02:45:30 on dist-test-slave-zqpb"
server_key: "5b435f609c5c91f1a0b4ea4116dc8f91"
server_key_iv: "2eb9aaf7c25205891adb8c282a780aaa"
server_key_version: "encryptionkey@0"
I20260619 02:45:30.918129 24059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:30.927327 24059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:30.928997 24059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:30.929111 24059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:30.929308 24059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:30.929589 24059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:30.929621 24059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:30.929642 24059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:30.929665 24059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:30.935096 24059 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:34501
I20260619 02:45:30.935137 24186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:34501 every 8 connection(s)
I20260619 02:45:30.935374 24059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:30.935783 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 24059
I20260619 02:45:30.935863 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:30.936020 19909 external_mini_cluster.cc:1428] Setting key 7169754ab676bbdb8a9ec06b3cf6a5bb
Jun 19 02:45:30 dist-test-slave-zqpb krb5kdc[23953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837130, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
I20260619 02:45:30.944782 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:30.936034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:51759 (local address 127.19.113.126:40031)
0619 02:45:30.936102 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:30.936104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:30.936311 (+ 207us) server_negotiation.cc:407] Connection header received
0619 02:45:30.936866 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:30.936867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:30.936878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:30.936927 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:30.937912 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:30.938408 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:30.939498 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:30.939791 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:30.939850 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:30.942120 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:30.942121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:30.942122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:30.942128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:30.943401 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:30.943821 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:30.943821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:30.943822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:30.943865 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:30.944136 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:30.944137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:30.944137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:30.944259 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:30.944322 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:30.944642 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 02:45:30.944718 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:45:30.944942 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:30.936022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40031 (local address 127.19.113.65:51759)
0619 02:45:30.936191 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:30.936237 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:30.936771 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:30.936996 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:30.937000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:30.937252 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:30.937806 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:30.937812 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:30.938499 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:30.938500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:30.939423 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:30.939425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:30.939872 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:30.939873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:30.940007 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:30.940476 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:30.940485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:30.942017 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:30.943475 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:30.943481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:30.943483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:30.943751 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:30.943917 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:30.943917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:30.943918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:30.944045 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:30.944325 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:30.944331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:30.944499 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 02:45:30.944611 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:45:30.944755 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260619 02:45:30.945401 24187 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40031
I20260619 02:45:30.945492 24187 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:30.945632 24187 heartbeater.cc:511] Master 127.19.113.126:40031 requested a full tablet report, sending...
I20260619 02:45:30.946192 23997 ts_manager.cc:194] Registered new tserver with Master: 6a2123cd853a4390953991128360c45f (127.19.113.65:34501)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
I20260619 02:45:30.946905 23997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:51759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:30.959666 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40031
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:31.040086 24192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:31.040287 24192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:31.040309 24192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:31.041447 24192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:31.041486 24192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:31.041510 24192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:31.043007 24192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:31.043213 24192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:31.043419 24192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:31.048085 24197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:31.048242 24192 server_base.cc:1034] running on GCE node
W20260619 02:45:31.048085 24200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.048126 24198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:31.048517 24192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:31.048748 24192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:31.049918 24192 hybrid_clock.cc:648] HybridClock initialized: now 1781837131049888 us; error 45 us; skew 500 ppm
Jun 19 02:45:31 dist-test-slave-zqpb krb5kdc[23953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:31.052279 24192 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:31.052594 24192 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:31.052642 24192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:31.052731 24192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:31.053737 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "18f4aaf1fd2b42b28dba1573fffa1f38"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "96ea4f2a8b8f9980fa150d0da760b479"
server_key_iv: "1f20e28db045632c61c799cdaeeeb01e"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.054049 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "18f4aaf1fd2b42b28dba1573fffa1f38"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "96ea4f2a8b8f9980fa150d0da760b479"
server_key_iv: "1f20e28db045632c61c799cdaeeeb01e"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.055251 24192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:31.055950 24206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.056118 24192 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:31.056180 24192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "18f4aaf1fd2b42b28dba1573fffa1f38"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "96ea4f2a8b8f9980fa150d0da760b479"
server_key_iv: "1f20e28db045632c61c799cdaeeeb01e"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.056242 24192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:31.065392 24192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:31.067405 24192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:31.067523 24192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:31.067734 24192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:31.068032 24192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:31.068064 24192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.068092 24192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:31.068116 24192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.073634 24192 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:40455
I20260619 02:45:31.073668 24319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:40455 every 8 connection(s)
I20260619 02:45:31.073904 24192 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:31.074108 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 24192
I20260619 02:45:31.074209 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:31.074404 19909 external_mini_cluster.cc:1428] Setting key bcc06500a1a5b3aad03f27278d4a9e53
Jun 19 02:45:31 dist-test-slave-zqpb krb5kdc[23953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
I20260619 02:45:31.083411 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.074645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:51669 (local address 127.19.113.126:40031)
0619 02:45:31.074706 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.074708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.074922 (+ 214us) server_negotiation.cc:407] Connection header received
0619 02:45:31.075434 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.075435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.075447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.075496 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:31.076444 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.076936 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.077924 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.078228 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.078303 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.080343 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:31.080345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:31.080346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:31.080355 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:31.081890 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:31.082256 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:31.082256 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:31.082257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:31.082339 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:31.082680 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:31.082681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:31.082681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:31.082813 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:31.082891 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.083226 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.083312 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:45:31.083520 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.074637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40031 (local address 127.19.113.66:51669)
0619 02:45:31.074794 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.074838 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.075348 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.075564 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.075568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.075816 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:31.076355 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.076360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.077015 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.077016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.077819 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.077822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.078320 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.078321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.078439 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.078983 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:31.078992 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:31.080252 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:31.081969 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:31.081976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:31.081978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:31.082191 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:31.082395 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:31.082396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:31.082396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:31.082589 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:31.082928 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:31.082933 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:31.083072 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.083173 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.083315 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
I20260619 02:45:31.083945 24320 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40031
I20260619 02:45:31.084028 24320 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:31.084169 24320 heartbeater.cc:511] Master 127.19.113.126:40031 requested a full tablet report, sending...
I20260619 02:45:31.084513 23997 ts_manager.cc:194] Registered new tserver with Master: 18f4aaf1fd2b42b28dba1573fffa1f38 (127.19.113.66:40455)
I20260619 02:45:31.084880 23997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:51669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:31.097564 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40031
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:31.176810 24325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:31.177032 24325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:31.177057 24325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:31.178215 24325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:31.178251 24325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:31.178277 24325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:31.179750 24325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:31.179952 24325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:31.180128 24325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:31.184823 24330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.184832 24333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:31.185055 24325 server_base.cc:1034] running on GCE node
W20260619 02:45:31.185161 24331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:31.185370 24325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:31.185578 24325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:31.186728 24325 hybrid_clock.cc:648] HybridClock initialized: now 1781837131186700 us; error 38 us; skew 500 ppm
Jun 19 02:45:31 dist-test-slave-zqpb krb5kdc[23953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:31.189256 24325 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:31.189632 24325 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:31.189685 24325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:31.189810 24325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:31.190937 24325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "6477f271d1d347cfa7cee3811ea1fded"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "a5019a4584ebb68e204bbfdf569746a8"
server_key_iv: "91cdd785b0fd3c25cd2372b1bb7319b8"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.191277 24325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "6477f271d1d347cfa7cee3811ea1fded"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "a5019a4584ebb68e204bbfdf569746a8"
server_key_iv: "91cdd785b0fd3c25cd2372b1bb7319b8"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.192523 24325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:31.193277 24339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.193449 24325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:31.193536 24325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "6477f271d1d347cfa7cee3811ea1fded"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "a5019a4584ebb68e204bbfdf569746a8"
server_key_iv: "91cdd785b0fd3c25cd2372b1bb7319b8"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.193603 24325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:31.207021 24325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:31.209290 24325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:31.209447 24325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:31.209666 24325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:31.209987 24325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:31.210031 24325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.210062 24325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:31.210078 24325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.215703 24325 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:40905
I20260619 02:45:31.215745 24452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:40905 every 8 connection(s)
I20260619 02:45:31.216014 24325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:45:31 dist-test-slave-zqpb krb5kdc[23953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:31.222230 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 24325
I20260619 02:45:31.222329 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:31.222507 19909 external_mini_cluster.cc:1428] Setting key 8f2bb06faec19ca40a6195f57cbd6c82
I20260619 02:45:31.225340 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.216660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:50147 (local address 127.19.113.126:40031)
0619 02:45:31.216705 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.216707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.216961 (+ 254us) server_negotiation.cc:407] Connection header received
0619 02:45:31.217465 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.217467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.217479 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.217528 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:31.218477 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.219029 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.220046 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.220249 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.220305 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.222552 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:31.222553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:31.222554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:31.222563 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:31.224005 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:31.224397 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:31.224399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:31.224399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:31.224458 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:31.224696 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:31.224697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:31.224698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:31.224826 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:31.224892 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.225166 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.225248 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:31.225550 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.216624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40031 (local address 127.19.113.67:50147)
0619 02:45:31.216796 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.216831 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.217364 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.217600 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.217605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.217881 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:31.218382 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.218387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.219121 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.219123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.219971 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.219974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.220319 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.220321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.220441 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.220954 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:31.220966 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:31.222451 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:31.224088 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:31.224095 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:31.224097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:31.224321 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:31.224518 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:31.224519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:31.224520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:31.224639 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:31.224891 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:31.224898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:31.225042 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.225134 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.225301 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260619 02:45:31.226011 24453 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40031
I20260619 02:45:31.226107 24453 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:31.226229 24453 heartbeater.cc:511] Master 127.19.113.126:40031 requested a full tablet report, sending...
I20260619 02:45:31.226549 23997 ts_manager.cc:194] Registered new tserver with Master: 6477f271d1d347cfa7cee3811ea1fded (127.19.113.67:40905)
I20260619 02:45:31.226935 23997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:50147
I20260619 02:45:31.234306 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:45:31.241741 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.235680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.19.113.126:40031)
0619 02:45:31.235730 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.235732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.235858 (+ 126us) server_negotiation.cc:407] Connection header received
0619 02:45:31.235918 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.235919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.235929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.235978 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:31.236559 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.237054 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.237786 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.237979 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.238031 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.239358 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:31.239359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:31.239360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:31.239367 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:31.240597 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:31.240985 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:31.240986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:31.240986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:31.241038 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:31.241209 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:31.241210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:31.241210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:31.241373 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:31.241464 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.241573 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.241650 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:45:31.243275 23997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:31.244856 23997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:31.250736 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.247025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36078 (local address 127.19.113.66:40455)
0619 02:45:31.247436 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.247441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.247450 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:31.247485 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.247487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.247502 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.247568 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:31.247772 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.248513 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.249512 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.250136 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.250197 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.250257 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.250600 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.250650 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":63,"threads_started":1}
I20260619 02:45:31.250734 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.246836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:40455 (local address 127.0.0.1:36078)
0619 02:45:31.247188 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.247230 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.247303 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.247570 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.247573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.247575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:31.247646 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.247648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.248607 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.248608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.249426 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.249428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.250382 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.250383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.250471 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.250486 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.250566 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.250633 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":104,"threads_started":1}
I20260619 02:45:31.251453 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.247053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:34501 (local address 127.0.0.1:44754)
0619 02:45:31.247362 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.247401 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.247454 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.247685 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.247686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.247689 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:31.247716 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.247717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.249051 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.249052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.250269 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.250272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.251202 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.251202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.251258 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.251261 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.251313 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.251349 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":49,"thread_start_us":76,"threads_started":1}
I20260619 02:45:31.251458 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.246837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40905 (local address 127.0.0.1:38458)
0619 02:45:31.247188 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.247229 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.247303 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.247655 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.247656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.247658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:31.247713 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.247715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.249053 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.249054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.250279 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.250280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.251188 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.251190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.251264 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.251267 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.251304 (+ 37us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.251349 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":38,"spinlock_wait_cycles":384,"thread_start_us":80,"threads_started":1}
I20260619 02:45:31.251471 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.247185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.19.113.65:34501)
0619 02:45:31.247358 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.247362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.247461 (+ 99us) server_negotiation.cc:407] Connection header received
0619 02:45:31.247518 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.247520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.247533 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.247599 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:31.247784 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.248944 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.250368 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.251063 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.251123 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.251174 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.251333 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.251388 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260619 02:45:31.251695 24254 tablet_service.cc:1467] Processing CreateTablet for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb (DEFAULT_TABLE table=test-table [id=f8eac32db9244d94ac35982cae4c99c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:31.252094 24254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:31.252296 24121 tablet_service.cc:1467] Processing CreateTablet for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb (DEFAULT_TABLE table=test-table [id=f8eac32db9244d94ac35982cae4c99c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:31.252573 24121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:31.254591 24467 tablet_bootstrap.cc:492] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Bootstrap starting.
I20260619 02:45:31.254776 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.246942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.19.113.67:40905)
0619 02:45:31.247208 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.247214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.247285 (+ 71us) server_negotiation.cc:407] Connection header received
0619 02:45:31.247400 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.247403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.247418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.247471 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:31.247794 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.248944 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.250359 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.251106 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.251172 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.254610 (+ 3438us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.254643 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.254698 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":99,"threads_started":1}
I20260619 02:45:31.255404 24467 tablet_bootstrap.cc:654] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:31.255453 24387 tablet_service.cc:1467] Processing CreateTablet for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb (DEFAULT_TABLE table=test-table [id=f8eac32db9244d94ac35982cae4c99c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:31.255784 24387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:31.255978 24467 log.cc:824] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:31.256734 24467 tablet_bootstrap.cc:492] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: No bootstrap required, opened a new log
I20260619 02:45:31.256831 24467 ts_tablet_manager.cc:1397] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:45:31.258827 24467 raft_consensus.cc:348] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.259027 24467 raft_consensus.cc:374] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:31.259073 24467 raft_consensus.cc:729] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f4aaf1fd2b42b28dba1573fffa1f38, State: Initialized, Role: FOLLOWER
I20260619 02:45:31.259079 24468 tablet_bootstrap.cc:492] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f: Bootstrap starting.
I20260619 02:45:31.259013 24470 tablet_bootstrap.cc:492] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Bootstrap starting.
I20260619 02:45:31.259225 24467 consensus_queue.cc:260] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.259518 24467 ts_tablet_manager.cc:1428] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:45:31.259545 24320 heartbeater.cc:503] Master 127.19.113.126:40031 was elected leader, sending a full tablet report...
I20260619 02:45:31.259723 24468 tablet_bootstrap.cc:654] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:31.260039 24470 tablet_bootstrap.cc:654] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:31.260286 24468 log.cc:824] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:31.260586 24470 log.cc:824] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:31.261039 24468 tablet_bootstrap.cc:492] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f: No bootstrap required, opened a new log
I20260619 02:45:31.261098 24468 ts_tablet_manager.cc:1397] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:31.262449 24468 raft_consensus.cc:348] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.262573 24468 raft_consensus.cc:374] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:31.262603 24468 raft_consensus.cc:729] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a2123cd853a4390953991128360c45f, State: Initialized, Role: FOLLOWER
I20260619 02:45:31.262717 24468 consensus_queue.cc:260] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.263032 24187 heartbeater.cc:503] Master 127.19.113.126:40031 was elected leader, sending a full tablet report...
I20260619 02:45:31.263037 24468 ts_tablet_manager.cc:1428] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:45:31.263530 24470 tablet_bootstrap.cc:492] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: No bootstrap required, opened a new log
I20260619 02:45:31.263600 24470 ts_tablet_manager.cc:1397] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260619 02:45:31.265057 24470 raft_consensus.cc:348] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.265182 24470 raft_consensus.cc:374] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:31.265203 24470 raft_consensus.cc:729] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6477f271d1d347cfa7cee3811ea1fded, State: Initialized, Role: FOLLOWER
I20260619 02:45:31.265297 24470 consensus_queue.cc:260] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.265542 24470 ts_tablet_manager.cc:1428] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:45:31.265568 24453 heartbeater.cc:503] Master 127.19.113.126:40031 was elected leader, sending a full tablet report...
W20260619 02:45:31.324486 24321 tablet.cc:2369] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:45:31.436084 24188 tablet.cc:2369] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:31.456951 24471 raft_consensus.cc:482] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:31.457100 24471 raft_consensus.cc:504] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.457465 24471 leader_election.cc:288] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6477f271d1d347cfa7cee3811ea1fded (127.19.113.67:40905), 6a2123cd853a4390953991128360c45f (127.19.113.65:34501)
I20260619 02:45:31.460412 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.457714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:55469 (local address 127.19.113.65:34501)
0619 02:45:31.457834 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.457836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.457847 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:31.457878 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.457879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.457892 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.457989 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:31.458126 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.458660 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.459706 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.460115 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.460177 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.460185 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.460305 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.460340 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 02:45:31.460445 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.457757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:40555 (local address 127.19.113.67:40905)
0619 02:45:31.457807 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.457809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.457925 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:45:31.457986 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.457987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.457996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.458119 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:31.458250 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.458757 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.459716 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.460139 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.460200 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.460208 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.460342 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.460376 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:45:31.460456 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.457652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:34501 (local address 127.19.113.66:55469)
0619 02:45:31.457714 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.457725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.457771 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.458006 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.458007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.458008 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:31.458065 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.458067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.458736 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.458737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.459625 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.459627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.460176 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.460177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.460249 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.460253 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.460295 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.460336 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 02:45:31.460484 24476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.457654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40905 (local address 127.19.113.66:40555)
0619 02:45:31.457856 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.457866 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.457926 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.458130 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.458131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.458132 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:31.458196 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.458197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.458829 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.458830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.459635 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:31.459637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.460206 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.460207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:31.460286 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.460289 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.460332 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.460369 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":15,"thread_start_us":104,"threads_started":1}
I20260619 02:45:31.460786 24407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b6a3f53ca8a4a56a77cbfcc671052bb" candidate_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6477f271d1d347cfa7cee3811ea1fded" is_pre_election: true
I20260619 02:45:31.460788 24141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b6a3f53ca8a4a56a77cbfcc671052bb" candidate_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a2123cd853a4390953991128360c45f" is_pre_election: true
I20260619 02:45:31.460947 24407 raft_consensus.cc:2454] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18f4aaf1fd2b42b28dba1573fffa1f38 in term 0.
I20260619 02:45:31.460947 24141 raft_consensus.cc:2454] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18f4aaf1fd2b42b28dba1573fffa1f38 in term 0.
I20260619 02:45:31.461169 24210 leader_election.cc:302] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f4aaf1fd2b42b28dba1573fffa1f38, 6477f271d1d347cfa7cee3811ea1fded; no voters:
I20260619 02:45:31.461342 24471 raft_consensus.cc:2781] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:31.461407 24471 raft_consensus.cc:482] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:31.461431 24471 raft_consensus.cc:3037] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:31.462081 24471 raft_consensus.cc:504] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.462213 24471 leader_election.cc:288] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [CANDIDATE]: Term 1 election: Requested vote from peers 6477f271d1d347cfa7cee3811ea1fded (127.19.113.67:40905), 6a2123cd853a4390953991128360c45f (127.19.113.65:34501)
I20260619 02:45:31.462368 24141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b6a3f53ca8a4a56a77cbfcc671052bb" candidate_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a2123cd853a4390953991128360c45f"
I20260619 02:45:31.462389 24407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b6a3f53ca8a4a56a77cbfcc671052bb" candidate_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6477f271d1d347cfa7cee3811ea1fded"
I20260619 02:45:31.462456 24407 raft_consensus.cc:3037] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:31.462451 24141 raft_consensus.cc:3037] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:31.463217 24407 raft_consensus.cc:2454] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f4aaf1fd2b42b28dba1573fffa1f38 in term 1.
I20260619 02:45:31.463277 24141 raft_consensus.cc:2454] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f4aaf1fd2b42b28dba1573fffa1f38 in term 1.
I20260619 02:45:31.463376 24210 leader_election.cc:302] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f4aaf1fd2b42b28dba1573fffa1f38, 6477f271d1d347cfa7cee3811ea1fded; no voters:
I20260619 02:45:31.463457 24471 raft_consensus.cc:2781] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:31.463593 24471 raft_consensus.cc:686] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 1 LEADER]: Becoming Leader. State: Replica: 18f4aaf1fd2b42b28dba1573fffa1f38, State: Running, Role: LEADER
I20260619 02:45:31.463685 24471 consensus_queue.cc:237] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } }
I20260619 02:45:31.464529 23997 catalog_manager.cc:5495] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18f4aaf1fd2b42b28dba1573fffa1f38 (127.19.113.66). New cstate: current_term: 1 leader_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18f4aaf1fd2b42b28dba1573fffa1f38" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 } health_report { overall_health: UNKNOWN } } }
W20260619 02:45:31.466476 24454 tablet.cc:2369] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:31.478017 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.475694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.19.113.66:40455)
0619 02:45:31.475760 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.475762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.475794 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:45:31.475828 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.475829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.475847 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.475899 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:31.476015 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.476480 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.477185 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:31.477365 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:31.477407 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:31.477666 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:31.477777 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:31.477851 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.477864 (+ 13us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.477939 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:45:31.479866 24407 raft_consensus.cc:1261] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 1 FOLLOWER]: Refusing update from remote peer 18f4aaf1fd2b42b28dba1573fffa1f38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:31.479866 24141 raft_consensus.cc:1261] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6a2123cd853a4390953991128360c45f [term 1 FOLLOWER]: Refusing update from remote peer 18f4aaf1fd2b42b28dba1573fffa1f38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:31.480296 24477 consensus_queue.cc:1035] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6477f271d1d347cfa7cee3811ea1fded" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:31.480424 24471 consensus_queue.cc:1035] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a2123cd853a4390953991128360c45f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:31.482918 24481 mvcc.cc:205] Tried to move back new op lower bound from 7298404890538541056 to 7298404890475798528. Current Snapshot: MvccSnapshot[applied={T|T < 7298404890538541056}]
I20260619 02:45:31.484583 24482 mvcc.cc:205] Tried to move back new op lower bound from 7298404890538541056 to 7298404890475798528. Current Snapshot: MvccSnapshot[applied={T|T < 7298404890538541056 or (T in {7298404890538541056})}]
I20260619 02:45:31.484676 23996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:31.484769 23996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:31.485478 23996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84ac1d5172c34bcc9e7ba9020d24343c: Sending DeleteTablet for 3 replicas of tablet 2b6a3f53ca8a4a56a77cbfcc671052bb
I20260619 02:45:31.485795 24387 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:31 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:38458
I20260619 02:45:31.485853 24254 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:31 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:36078
I20260619 02:45:31.486034 24489 tablet_replica.cc:330] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: stopping tablet replica
I20260619 02:45:31.486120 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 24059
I20260619 02:45:31.486135 24489 raft_consensus.cc:2229] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:31.486056 24121 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:31 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:44754
I20260619 02:45:31.486235 24489 raft_consensus.cc:2258] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:31.486670 24490 tablet_replica.cc:330] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: stopping tablet replica
I20260619 02:45:31.486783 24490 raft_consensus.cc:2229] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:31.486924 24490 raft_consensus.cc:2258] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:31.487341 24490 ts_tablet_manager.cc:1905] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:31.487780 24489 ts_tablet_manager.cc:1905] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:31.489014 24490 ts_tablet_manager.cc:1918] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:31.489084 24490 log.cc:1196] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/2b6a3f53ca8a4a56a77cbfcc671052bb
I20260619 02:45:31.489343 24490 ts_tablet_manager.cc:1939] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 18f4aaf1fd2b42b28dba1573fffa1f38: Deleting consensus metadata
I20260619 02:45:31.489562 24489 ts_tablet_manager.cc:1918] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:31.489691 24489 log.cc:1196] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/2b6a3f53ca8a4a56a77cbfcc671052bb
I20260619 02:45:31.490023 24489 ts_tablet_manager.cc:1939] T 2b6a3f53ca8a4a56a77cbfcc671052bb P 6477f271d1d347cfa7cee3811ea1fded: Deleting consensus metadata
I20260619 02:45:31.490499 23984 catalog_manager.cc:4841] TS 6477f271d1d347cfa7cee3811ea1fded (127.19.113.67:40905): tablet 2b6a3f53ca8a4a56a77cbfcc671052bb (table test-table [id=f8eac32db9244d94ac35982cae4c99c6]) successfully deleted
I20260619 02:45:31.491025 23983 catalog_manager.cc:4841] TS 18f4aaf1fd2b42b28dba1573fffa1f38 (127.19.113.66:40455): tablet 2b6a3f53ca8a4a56a77cbfcc671052bb (table test-table [id=f8eac32db9244d94ac35982cae4c99c6]) successfully deleted
W20260619 02:45:31.492089 23983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:34501 (error 108)
I20260619 02:45:31.492307 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 24192
I20260619 02:45:31.492908 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.492416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:34501 (local address 127.0.0.1:44770)
0619 02:45:31.492833 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.492873 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397}
W20260619 02:45:31.493037 23983 catalog_manager.cc:4584] TS 6a2123cd853a4390953991128360c45f (127.19.113.65:34501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b6a3f53ca8a4a56a77cbfcc671052bb: Network error: Client connection negotiation failed: client connection to 127.19.113.65:34501: connect: Connection refused (error 111)
I20260619 02:45:31.496811 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 24325
I20260619 02:45:31.501464 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 23966
2026-06-19T02:45:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1931 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T02:45:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:31Z Disabled control of system clock
I20260619 02:45:31.526710 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40379
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:40379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:45:31.607262 24495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:31.607502 24495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:31.607527 24495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:31.608865 24495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:31.608914 24495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:31.608925 24495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:31.608937 24495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:31.608947 24495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:31.610368 24495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:40379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:31.610603 24495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:31.610906 24495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:31.615885 24500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.615993 24501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.615885 24503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:31.616066 24495 server_base.cc:1034] running on GCE node
I20260619 02:45:31.616307 24495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:31.616575 24495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:31.617758 24495 hybrid_clock.cc:648] HybridClock initialized: now 1781837131617740 us; error 27 us; skew 500 ppm
I20260619 02:45:31.619114 24495 webserver.cc:460] Webserver started at http://127.19.113.126:36619/ using document root <none> and password file <none>
I20260619 02:45:31.619364 24495 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:31.619421 24495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:31.619549 24495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:31.620728 24495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "7161129e5a8b4f6f8e0c88fbb10307b0"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "471b26fe4b291c2934a79eaa1c3df16d"
server_key_iv: "887d2e9426e648152957d3389554e8a0"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.621173 24495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "7161129e5a8b4f6f8e0c88fbb10307b0"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "471b26fe4b291c2934a79eaa1c3df16d"
server_key_iv: "887d2e9426e648152957d3389554e8a0"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.622540 24495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:31.623515 24509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.623737 24495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:31.623823 24495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "7161129e5a8b4f6f8e0c88fbb10307b0"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "471b26fe4b291c2934a79eaa1c3df16d"
server_key_iv: "887d2e9426e648152957d3389554e8a0"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.623953 24495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:31.627369 24495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:31.627638 24495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:31.627756 24495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:31.631467 24495 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:40379
I20260619 02:45:31.631506 24561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:40379 every 8 connection(s)
I20260619 02:45:31.631825 24495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:31.632459 24562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:31.635124 24562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0: Bootstrap starting.
I20260619 02:45:31.635644 24562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:31.635865 24562 log.cc:824] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:31.636480 24562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0: No bootstrap required, opened a new log
I20260619 02:45:31.638008 24562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7161129e5a8b4f6f8e0c88fbb10307b0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40379 } }
I20260619 02:45:31.638141 24562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:31.638161 24562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7161129e5a8b4f6f8e0c88fbb10307b0, State: Initialized, Role: FOLLOWER
I20260619 02:45:31.638259 24562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7161129e5a8b4f6f8e0c88fbb10307b0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40379 } }
I20260619 02:45:31.638309 24562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:31.638327 24562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:31.638397 24562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:31.638940 24562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7161129e5a8b4f6f8e0c88fbb10307b0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40379 } }
I20260619 02:45:31.639030 24562 leader_election.cc:302] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7161129e5a8b4f6f8e0c88fbb10307b0; no voters:
I20260619 02:45:31.639173 24562 leader_election.cc:288] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:31.639240 24565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:31.639391 24565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [term 1 LEADER]: Becoming Leader. State: Replica: 7161129e5a8b4f6f8e0c88fbb10307b0, State: Running, Role: LEADER
I20260619 02:45:31.639415 24562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:31.639478 24565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7161129e5a8b4f6f8e0c88fbb10307b0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40379 } }
I20260619 02:45:31.639855 24567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7161129e5a8b4f6f8e0c88fbb10307b0. Latest consensus state: current_term: 1 leader_uuid: "7161129e5a8b4f6f8e0c88fbb10307b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7161129e5a8b4f6f8e0c88fbb10307b0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40379 } } }
I20260619 02:45:31.640000 24567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:31.640177 24566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7161129e5a8b4f6f8e0c88fbb10307b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7161129e5a8b4f6f8e0c88fbb10307b0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40379 } } }
I20260619 02:45:31.640316 24566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:31.640244 24574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:31.640859 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 24495
I20260619 02:45:31.640918 24574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:31.640972 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:31.641149 19909 external_mini_cluster.cc:1428] Setting key 6d310cd4610336031e8db4803617db47
I20260619 02:45:31.642683 24574 catalog_manager.cc:1347] Generated new cluster ID: cef8dc7a2b8d40808960f1603279ce7e
I20260619 02:45:31.642735 24574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:31.648020 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.641716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35790 (local address 127.19.113.126:40379)
0619 02:45:31.642068 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.642077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.642106 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:45:31.642216 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.642223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.642268 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.647197 (+ 4929us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:31.647528 (+ 331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:31.647531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:31.647531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:31.647534 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:31.647563 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:31.647581 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:31.647646 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.647722 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.647862 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":73,"threads_started":1}
I20260619 02:45:31.653968 24574 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:31.654136 24574 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:31.660987 24574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0: Generated new TSK 0
I20260619 02:45:31.661222 24574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:31.667461 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40379
--builtin_ntp_servers=127.19.113.84:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:45:31.750267 24586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:31.750635 24586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:31.750692 24586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:31.752441 24586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:31.752496 24586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:31.752540 24586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:31.754226 24586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:31.754510 24586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:31.754828 24586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:31.759948 24592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.759958 24594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.759955 24591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:31.760672 24586 server_base.cc:1034] running on GCE node
I20260619 02:45:31.760931 24586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:31.761178 24586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:31.762369 24586 hybrid_clock.cc:648] HybridClock initialized: now 1781837131762355 us; error 28 us; skew 500 ppm
I20260619 02:45:31.763785 24586 webserver.cc:460] Webserver started at http://127.19.113.65:34717/ using document root <none> and password file <none>
I20260619 02:45:31.764035 24586 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:31.764081 24586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:31.764209 24586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:31.765295 24586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "e9f786bdb31516b8937502fa3eef8985"
server_key_iv: "ba9007def61bfc33bbae38d1743c624a"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.765694 24586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "e9f786bdb31516b8937502fa3eef8985"
server_key_iv: "ba9007def61bfc33bbae38d1743c624a"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.767079 24586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:45:31.767794 24600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.767992 24586 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:31.768076 24586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "e9f786bdb31516b8937502fa3eef8985"
server_key_iv: "ba9007def61bfc33bbae38d1743c624a"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.768133 24586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:31.771478 24586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:31.771708 24586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:31.771809 24586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:31.772012 24586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:31.772286 24586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:31.772316 24586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.772334 24586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:31.772357 24586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.777858 24586 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:38605
I20260619 02:45:31.777972 24713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:38605 every 8 connection(s)
I20260619 02:45:31.778357 24586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:31.781751 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.779426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:44847 (local address 127.19.113.126:40379)
0619 02:45:31.779510 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.779511 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.779739 (+ 228us) server_negotiation.cc:407] Connection header received
0619 02:45:31.780300 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.780302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.780310 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.780361 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:31.781510 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:31.781512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:31.781512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:31.781517 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:31.781534 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:31.781539 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:31.781588 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.781663 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.781677 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:45:31.781952 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.779408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40379 (local address 127.19.113.65:44847)
0619 02:45:31.779613 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.779649 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.780192 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.780432 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.780437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.780447 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:31.781352 (+ 905us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:45:31.781353 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:31.781384 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:45:31.781416 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:31.781591 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:31.781597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:31.781600 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.781671 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.781734 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260619 02:45:31.782296 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 24586
I20260619 02:45:31.782415 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:31.782595 19909 external_mini_cluster.cc:1428] Setting key c3ddac97993f3c92b95f28d014c5a3af
I20260619 02:45:31.782620 24714 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40379
I20260619 02:45:31.782716 24714 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:31.782847 24714 heartbeater.cc:511] Master 127.19.113.126:40379 requested a full tablet report, sending...
I20260619 02:45:31.783613 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40379
--builtin_ntp_servers=127.19.113.84:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 02:45:31.783792 24526 ts_manager.cc:194] Registered new tserver with Master: 80e31a6c217a4e9aa3ba33b0f2cf711f (127.19.113.65:38605)
W20260619 02:45:31.866266 24717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:31.866477 24717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:31.866513 24717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:31.867776 24717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:31.867822 24717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:31.867848 24717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:31.869233 24717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:31.869472 24717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:31.869766 24717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:31.875088 24725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.875111 24722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.875098 24723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:31.875174 24717 server_base.cc:1034] running on GCE node
I20260619 02:45:31.875533 24717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:31.875808 24717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:31.876956 24717 hybrid_clock.cc:648] HybridClock initialized: now 1781837131876936 us; error 38 us; skew 500 ppm
I20260619 02:45:31.878196 24717 webserver.cc:460] Webserver started at http://127.19.113.66:40833/ using document root <none> and password file <none>
I20260619 02:45:31.878423 24717 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:31.878497 24717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:31.878639 24717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:31.879758 24717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "7fee5c7e75524dcf9d0ffae8edb3122f"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "b9df7c2a6ac5b36a456fc214ae403922"
server_key_iv: "ccb58ffad8c9fe1be766be10a494812e"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.880085 24717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "7fee5c7e75524dcf9d0ffae8edb3122f"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "b9df7c2a6ac5b36a456fc214ae403922"
server_key_iv: "ccb58ffad8c9fe1be766be10a494812e"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.881314 24717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:45:31.882134 24731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.882310 24717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:31.882421 24717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "7fee5c7e75524dcf9d0ffae8edb3122f"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "b9df7c2a6ac5b36a456fc214ae403922"
server_key_iv: "ccb58ffad8c9fe1be766be10a494812e"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.882483 24717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:31.885715 24717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:31.885931 24717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:31.886021 24717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:31.886171 24717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:31.886425 24717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:31.886456 24717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.886475 24717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:31.886487 24717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.892251 24717 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:37117
I20260619 02:45:31.892287 24844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:37117 every 8 connection(s)
I20260619 02:45:31.892617 24717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:31.895792 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.893377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:41869 (local address 127.19.113.126:40379)
0619 02:45:31.893435 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:31.893437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:31.893666 (+ 229us) server_negotiation.cc:407] Connection header received
0619 02:45:31.894184 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:31.894185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:31.894192 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:31.894244 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:31.895556 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:31.895558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:31.895558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:31.895563 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:31.895581 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:31.895586 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:31.895638 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:31.895704 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 02:45:31.895718 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:45:31.895998 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:31.893377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40379 (local address 127.19.113.66:41869)
0619 02:45:31.893552 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:31.893587 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:31.894088 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:31.894321 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:31.894326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:31.894336 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:31.895389 (+ 1053us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:45:31.895389 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:31.895419 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:45:31.895448 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:31.895639 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:31.895644 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:31.895648 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:31.895716 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:45:31.895779 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260619 02:45:31.896456 24845 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40379
I20260619 02:45:31.896538 24845 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:31.896683 24845 heartbeater.cc:511] Master 127.19.113.126:40379 requested a full tablet report, sending...
I20260619 02:45:31.897157 24526 ts_manager.cc:194] Registered new tserver with Master: 7fee5c7e75524dcf9d0ffae8edb3122f (127.19.113.66:37117)
I20260619 02:45:31.898528 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 24717
I20260619 02:45:31.898620 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:31.898789 19909 external_mini_cluster.cc:1428] Setting key 93f5560040ef99406f45e83e846a1308
I20260619 02:45:31.899677 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40379
--builtin_ntp_servers=127.19.113.84:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:45:31.979693 24848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:31.979899 24848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:31.979921 24848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:31.981045 24848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:31.981081 24848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:31.981105 24848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:31.982422 24848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:31.982620 24848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:31.982859 24848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:31.987912 24853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.987952 24856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:31.987938 24854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:31.988099 24848 server_base.cc:1034] running on GCE node
I20260619 02:45:31.988400 24848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:31.988662 24848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:31.989858 24848 hybrid_clock.cc:648] HybridClock initialized: now 1781837131989845 us; error 32 us; skew 500 ppm
I20260619 02:45:31.991451 24848 webserver.cc:460] Webserver started at http://127.19.113.67:35947/ using document root <none> and password file <none>
I20260619 02:45:31.991704 24848 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:31.991756 24848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:31.991889 24848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:31.993014 24848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "ee455c7e09f94f7498c02ab46f38bdbf"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "9f40c5a6db679c2f93c8c90ac8bb678e"
server_key_iv: "abe1fcfd6a4cad24036b93892f69ed08"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.993501 24848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "ee455c7e09f94f7498c02ab46f38bdbf"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "9f40c5a6db679c2f93c8c90ac8bb678e"
server_key_iv: "abe1fcfd6a4cad24036b93892f69ed08"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.995359 24848 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 02:45:31.996249 24862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:31.996534 24848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:31.996637 24848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "ee455c7e09f94f7498c02ab46f38bdbf"
format_stamp: "Formatted at 2026-06-19 02:45:31 on dist-test-slave-zqpb"
server_key: "9f40c5a6db679c2f93c8c90ac8bb678e"
server_key_iv: "abe1fcfd6a4cad24036b93892f69ed08"
server_key_version: "encryptionkey@0"
I20260619 02:45:31.996717 24848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:32.000415 24848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:32.000695 24848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:32.000815 24848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:32.001003 24848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:32.001292 24848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:32.001325 24848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:32.001355 24848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:32.001379 24848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:32.006927 24848 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:35953
I20260619 02:45:32.007026 24975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:35953 every 8 connection(s)
I20260619 02:45:32.007325 24848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:32.010222 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.008050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:54201 (local address 127.19.113.126:40379)
0619 02:45:32.008101 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.008103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.008317 (+ 214us) server_negotiation.cc:407] Connection header received
0619 02:45:32.008797 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.008799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.008806 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.008873 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.010026 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.010028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.010028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.010031 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.010045 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.010049 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.010105 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.010153 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.010165 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:45:32.010448 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.008019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40379 (local address 127.19.113.67:54201)
0619 02:45:32.008202 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.008243 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.008689 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.008934 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.008938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.008945 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:32.009882 (+ 937us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:45:32.009882 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:32.009913 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:45:32.009942 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:32.010106 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:32.010108 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:32.010111 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:32.010160 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:45:32.010208 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260619 02:45:32.011735 24976 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40379
I20260619 02:45:32.011824 24976 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:32.012014 24976 heartbeater.cc:511] Master 127.19.113.126:40379 requested a full tablet report, sending...
I20260619 02:45:32.012387 24526 ts_manager.cc:194] Registered new tserver with Master: ee455c7e09f94f7498c02ab46f38bdbf (127.19.113.67:35953)
I20260619 02:45:32.014271 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 24848
I20260619 02:45:32.014362 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:32.014496 19909 external_mini_cluster.cc:1428] Setting key b56aef8cf14db605b9e2e320e2914da4
I20260619 02:45:32.015303 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:45:32.017568 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.016544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.19.113.126:40379)
0619 02:45:32.016581 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.016583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.016706 (+ 123us) server_negotiation.cc:407] Connection header received
0619 02:45:32.016773 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.016775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.016782 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.016826 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.017356 (+ 530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.017357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.017357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.017361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.017380 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.017384 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.017429 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.017476 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.017492 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 02:45:32.019071 24526 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:32.020583 24526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:32.024015 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.022575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:35953 (local address 127.0.0.1:56428)
0619 02:45:32.022751 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.022790 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.022843 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.023079 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.023085 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.023091 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:32.023566 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:45:32.023567 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:32.023583 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:45:32.023608 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:32.023853 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:32.023860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:32.023862 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:45:32.023932 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:45:32.023946 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":52,"threads_started":1}
I20260619 02:45:32.024216 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.022721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.19.113.67:35953)
0619 02:45:32.022944 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.022949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.022960 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:32.022985 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.022987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.022997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.023070 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.023692 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.023701 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.023702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.023706 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.023741 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.023768 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.023826 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.024122 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.024139 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
I20260619 02:45:32.024545 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.022582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:37117 (local address 127.0.0.1:44502)
0619 02:45:32.023037 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.023045 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.023088 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.023541 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.023542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.023545 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:32.024001 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:45:32.024001 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:32.024009 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:45:32.024021 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:32.024421 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:32.024422 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:32.024423 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:45:32.024471 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:45:32.024483 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":101,"threads_started":1}
I20260619 02:45:32.024719 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.022828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44502 (local address 127.19.113.66:37117)
0619 02:45:32.023026 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.023029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.023086 (+ 57us) server_negotiation.cc:407] Connection header received
0619 02:45:32.023133 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.023135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.023144 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.023211 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.024256 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.024262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.024263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.024266 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.024299 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.024323 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.024393 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.024488 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.024506 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260619 02:45:32.025077 24910 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdce02b51a14e6d8f75017e8ed6648e (DEFAULT_TABLE table=test-table [id=02f53e2d966d4064952ceb277f1679b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:32.025378 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.022582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:38605 (local address 127.0.0.1:46222)
0619 02:45:32.022951 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.022960 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.022990 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.023708 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.023709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.023712 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:32.024118 (+ 406us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:45:32.024119 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:32.024124 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:45:32.024133 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:32.024347 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:32.024348 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:32.024349 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:45:32.025298 (+ 949us) client_negotiation.cc:240] Negotiation successful
0619 02:45:32.025314 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":101,"threads_started":1}
I20260619 02:45:32.025453 24910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdce02b51a14e6d8f75017e8ed6648e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:32.025373 24779 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdce02b51a14e6d8f75017e8ed6648e (DEFAULT_TABLE table=test-table [id=02f53e2d966d4064952ceb277f1679b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:32.025646 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.022839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46222 (local address 127.19.113.65:38605)
0619 02:45:32.023316 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.023320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.023331 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:32.023356 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.023358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.023368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.023434 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.024204 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.024211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.024212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.024216 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.024246 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.024273 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.024325 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.025545 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.025563 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":96,"threads_started":1}
I20260619 02:45:32.025717 24779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdce02b51a14e6d8f75017e8ed6648e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:32.026271 24648 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdce02b51a14e6d8f75017e8ed6648e (DEFAULT_TABLE table=test-table [id=02f53e2d966d4064952ceb277f1679b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:32.026610 24648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdce02b51a14e6d8f75017e8ed6648e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:32.028231 24990 tablet_bootstrap.cc:492] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Bootstrap starting.
I20260619 02:45:32.028894 24990 tablet_bootstrap.cc:654] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:32.028928 24991 tablet_bootstrap.cc:492] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Bootstrap starting.
I20260619 02:45:32.029194 24990 log.cc:824] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:32.029511 24991 tablet_bootstrap.cc:654] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:32.029713 24993 tablet_bootstrap.cc:492] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f: Bootstrap starting.
I20260619 02:45:32.029805 24991 log.cc:824] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:32.030400 24993 tablet_bootstrap.cc:654] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:32.030421 24990 tablet_bootstrap.cc:492] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: No bootstrap required, opened a new log
I20260619 02:45:32.030544 24991 tablet_bootstrap.cc:492] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: No bootstrap required, opened a new log
I20260619 02:45:32.030612 24991 ts_tablet_manager.cc:1397] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:32.030623 24990 ts_tablet_manager.cc:1397] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:32.030712 24993 log.cc:824] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:32.031481 24993 tablet_bootstrap.cc:492] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f: No bootstrap required, opened a new log
I20260619 02:45:32.031550 24993 ts_tablet_manager.cc:1397] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:32.032691 24990 raft_consensus.cc:348] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.032832 24990 raft_consensus.cc:374] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:32.032862 24990 raft_consensus.cc:729] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee455c7e09f94f7498c02ab46f38bdbf, State: Initialized, Role: FOLLOWER
I20260619 02:45:32.033072 24990 consensus_queue.cc:260] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.033185 24991 raft_consensus.cc:348] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.033385 24990 ts_tablet_manager.cc:1428] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:45:32.033452 24976 heartbeater.cc:503] Master 127.19.113.126:40379 was elected leader, sending a full tablet report...
I20260619 02:45:32.033381 24991 raft_consensus.cc:374] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:32.033728 24991 raft_consensus.cc:729] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fee5c7e75524dcf9d0ffae8edb3122f, State: Initialized, Role: FOLLOWER
I20260619 02:45:32.033893 24991 consensus_queue.cc:260] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.034192 24845 heartbeater.cc:503] Master 127.19.113.126:40379 was elected leader, sending a full tablet report...
I20260619 02:45:32.034271 24993 raft_consensus.cc:348] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.034415 24993 raft_consensus.cc:374] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:32.034451 24993 raft_consensus.cc:729] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e31a6c217a4e9aa3ba33b0f2cf711f, State: Initialized, Role: FOLLOWER
I20260619 02:45:32.034598 24991 ts_tablet_manager.cc:1428] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 02:45:32.034614 24993 consensus_queue.cc:260] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.034861 24993 ts_tablet_manager.cc:1428] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:45:32.035100 24714 heartbeater.cc:503] Master 127.19.113.126:40379 was elected leader, sending a full tablet report...
W20260619 02:45:32.143124 24846 tablet.cc:2369] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:45:32.257880 24977 tablet.cc:2369] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:32.268873 24998 raft_consensus.cc:482] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:32.269014 24998 raft_consensus.cc:504] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.269397 24998 leader_election.cc:288] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee455c7e09f94f7498c02ab46f38bdbf (127.19.113.67:35953), 7fee5c7e75524dcf9d0ffae8edb3122f (127.19.113.66:37117)
I20260619 02:45:32.270687 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.269586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:35953 (local address 127.19.113.65:51673)
0619 02:45:32.269632 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.269644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.269687 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.269902 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.269903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.269905 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:32.270368 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:45:32.270369 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:32.270378 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:45:32.270392 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:32.270550 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:32.270551 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:32.270552 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:45:32.270593 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 02:45:32.270606 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:45:32.270704 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.269745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:51673 (local address 127.19.113.67:35953)
0619 02:45:32.269785 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.269787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.269795 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:32.269833 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.269834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.269838 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.269883 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.270461 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.270462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.270462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.270466 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.270482 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.270486 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.270554 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.270619 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.270634 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:32.271039 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.269611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:37117 (local address 127.19.113.65:51459)
0619 02:45:32.269754 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.269762 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.269813 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.269990 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.269991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.269993 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:32.270605 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:45:32.270606 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:32.270611 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:45:32.270621 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:32.270900 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:32.270901 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:32.270902 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:45:32.270951 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:45:32.270965 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260619 02:45:32.271060 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.269652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:51459 (local address 127.19.113.66:37117)
0619 02:45:32.269694 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.269696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.269829 (+ 133us) server_negotiation.cc:407] Connection header received
0619 02:45:32.269880 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.269882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.269887 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.269978 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.270683 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.270684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.270684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.270689 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.270705 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.270709 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.270753 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.270995 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.271011 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:32.271090 24930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdce02b51a14e6d8f75017e8ed6648e" candidate_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" is_pre_election: true
I20260619 02:45:32.271237 24930 raft_consensus.cc:2454] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80e31a6c217a4e9aa3ba33b0f2cf711f in term 0.
I20260619 02:45:32.271504 24603 leader_election.cc:302] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80e31a6c217a4e9aa3ba33b0f2cf711f, ee455c7e09f94f7498c02ab46f38bdbf; no voters:
I20260619 02:45:32.271646 24799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdce02b51a14e6d8f75017e8ed6648e" candidate_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" is_pre_election: true
I20260619 02:45:32.271708 24998 raft_consensus.cc:2781] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:32.271770 24799 raft_consensus.cc:2454] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80e31a6c217a4e9aa3ba33b0f2cf711f in term 0.
I20260619 02:45:32.271781 24998 raft_consensus.cc:482] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:32.271813 24998 raft_consensus.cc:3037] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:32.272537 24998 raft_consensus.cc:504] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.272660 24998 leader_election.cc:288] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [CANDIDATE]: Term 1 election: Requested vote from peers ee455c7e09f94f7498c02ab46f38bdbf (127.19.113.67:35953), 7fee5c7e75524dcf9d0ffae8edb3122f (127.19.113.66:37117)
I20260619 02:45:32.272871 24930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdce02b51a14e6d8f75017e8ed6648e" candidate_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee455c7e09f94f7498c02ab46f38bdbf"
I20260619 02:45:32.272840 24799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdce02b51a14e6d8f75017e8ed6648e" candidate_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f"
I20260619 02:45:32.272953 24930 raft_consensus.cc:3037] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:32.272965 24799 raft_consensus.cc:3037] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:32.273825 24799 raft_consensus.cc:2454] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80e31a6c217a4e9aa3ba33b0f2cf711f in term 1.
I20260619 02:45:32.273825 24930 raft_consensus.cc:2454] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80e31a6c217a4e9aa3ba33b0f2cf711f in term 1.
I20260619 02:45:32.274030 24603 leader_election.cc:302] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80e31a6c217a4e9aa3ba33b0f2cf711f, ee455c7e09f94f7498c02ab46f38bdbf; no voters:
I20260619 02:45:32.274129 24998 raft_consensus.cc:2781] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:32.274287 24998 raft_consensus.cc:686] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [term 1 LEADER]: Becoming Leader. State: Replica: 80e31a6c217a4e9aa3ba33b0f2cf711f, State: Running, Role: LEADER
I20260619 02:45:32.274394 24998 consensus_queue.cc:237] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } }
I20260619 02:45:32.275187 24526 catalog_manager.cc:5495] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f reported cstate change: term changed from 0 to 1, leader changed from <none> to 80e31a6c217a4e9aa3ba33b0f2cf711f (127.19.113.65). New cstate: current_term: 1 leader_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e31a6c217a4e9aa3ba33b0f2cf711f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 38605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 } health_report { overall_health: UNKNOWN } } }
W20260619 02:45:32.278986 24715 tablet.cc:2369] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:32.308619 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.307708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46228 (local address 127.19.113.65:38605)
0619 02:45:32.307751 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.307752 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.307759 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:32.307808 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.307809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.307821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.307859 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.308454 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.308456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.308456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.308460 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.308475 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.308479 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.308522 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.308555 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.308565 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:32.310319 24930 raft_consensus.cc:1261] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 1 FOLLOWER]: Refusing update from remote peer 80e31a6c217a4e9aa3ba33b0f2cf711f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:32.310320 24799 raft_consensus.cc:1261] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 1 FOLLOWER]: Refusing update from remote peer 80e31a6c217a4e9aa3ba33b0f2cf711f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:32.310684 25000 consensus_queue.cc:1035] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee455c7e09f94f7498c02ab46f38bdbf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:32.310811 24998 consensus_queue.cc:1035] T 1bdce02b51a14e6d8f75017e8ed6648e P 80e31a6c217a4e9aa3ba33b0f2cf711f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fee5c7e75524dcf9d0ffae8edb3122f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:32.313038 25003 mvcc.cc:205] Tried to move back new op lower bound from 7298404893940678656 to 7298404893796388864. Current Snapshot: MvccSnapshot[applied={T|T < 7298404893940678656}]
I20260619 02:45:32.313282 25005 mvcc.cc:205] Tried to move back new op lower bound from 7298404893940678656 to 7298404893796388864. Current Snapshot: MvccSnapshot[applied={T|T < 7298404893940678656}]
I20260619 02:45:32.314796 25004 mvcc.cc:205] Tried to move back new op lower bound from 7298404893940678656 to 7298404893796388864. Current Snapshot: MvccSnapshot[applied={T|T < 7298404893940678656 or (T in {7298404893940678656})}]
I20260619 02:45:32.315094 24526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:32.315167 24526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:32.315989 24526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7161129e5a8b4f6f8e0c88fbb10307b0: Sending DeleteTablet for 3 replicas of tablet 1bdce02b51a14e6d8f75017e8ed6648e
I20260619 02:45:32.316315 24910 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdce02b51a14e6d8f75017e8ed6648e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:32 UTC) from {username='slave'} at 127.0.0.1:56428
I20260619 02:45:32.316329 24648 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdce02b51a14e6d8f75017e8ed6648e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:32 UTC) from {username='slave'} at 127.0.0.1:46222
I20260619 02:45:32.316586 24779 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdce02b51a14e6d8f75017e8ed6648e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:32 UTC) from {username='slave'} at 127.0.0.1:44502
I20260619 02:45:32.316622 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 24586
I20260619 02:45:32.317113 25015 tablet_replica.cc:330] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: stopping tablet replica
I20260619 02:45:32.317204 25015 raft_consensus.cc:2229] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:32.317293 25015 raft_consensus.cc:2258] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:32.317473 25013 tablet_replica.cc:330] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: stopping tablet replica
I20260619 02:45:32.317570 25013 raft_consensus.cc:2229] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:32.317657 25013 raft_consensus.cc:2258] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:32.317790 25015 ts_tablet_manager.cc:1905] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:32.319315 25015 ts_tablet_manager.cc:1918] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:32.319381 25015 log.cc:1196] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/1bdce02b51a14e6d8f75017e8ed6648e
I20260619 02:45:32.319595 25015 ts_tablet_manager.cc:1939] T 1bdce02b51a14e6d8f75017e8ed6648e P 7fee5c7e75524dcf9d0ffae8edb3122f: Deleting consensus metadata
I20260619 02:45:32.319973 24510 catalog_manager.cc:4841] TS 7fee5c7e75524dcf9d0ffae8edb3122f (127.19.113.66:37117): tablet 1bdce02b51a14e6d8f75017e8ed6648e (table test-table [id=02f53e2d966d4064952ceb277f1679b4]) successfully deleted
W20260619 02:45:32.321966 24511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:38605 (error 108)
I20260619 02:45:32.322211 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 24717
I20260619 02:45:32.322491 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.322375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:38605 (local address 127.0.0.1:46232)
0619 02:45:32.322422 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.322460 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:38605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:45:32.322484 25013 ts_tablet_manager.cc:1905] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:45:32.322582 24511 catalog_manager.cc:4584] TS 80e31a6c217a4e9aa3ba33b0f2cf711f (127.19.113.65:38605): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1bdce02b51a14e6d8f75017e8ed6648e: Network error: Client connection negotiation failed: client connection to 127.19.113.65:38605: connect: Connection refused (error 111)
I20260619 02:45:32.324157 25013 ts_tablet_manager.cc:1918] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:32.324239 25013 log.cc:1196] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/1bdce02b51a14e6d8f75017e8ed6648e
I20260619 02:45:32.324467 25013 ts_tablet_manager.cc:1939] T 1bdce02b51a14e6d8f75017e8ed6648e P ee455c7e09f94f7498c02ab46f38bdbf: Deleting consensus metadata
I20260619 02:45:32.324873 24512 catalog_manager.cc:4841] TS ee455c7e09f94f7498c02ab46f38bdbf (127.19.113.67:35953): tablet 1bdce02b51a14e6d8f75017e8ed6648e (table test-table [id=02f53e2d966d4064952ceb277f1679b4]) successfully deleted
I20260619 02:45:32.327136 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 24848
I20260619 02:45:32.331807 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 24495
2026-06-19T02:45:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (829 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 02:45:32.350122 19909 mini_oidc.cc:152] Starting JWKS server
I20260619 02:45:32.351202 19909 webserver.cc:460] Webserver started at https://127.0.0.1:38511/ using document root <none> and password file <none>
I20260619 02:45:32.351279 19909 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:45:32.351419 19909 webserver.cc:460] Webserver started at http://127.0.0.1:46689/ using document root <none> and password file <none>
2026-06-19T02:45:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:32Z Disabled control of system clock
I20260619 02:45:32.359540 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40981
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:40337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:40981
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38511/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/testchainca.cert with env {}
W20260619 02:45:32.439606 25022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:32.439837 25022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:32.439862 25022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:32.441098 25022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:32.441139 25022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:32.441151 25022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:32.441162 25022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:32.441172 25022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:45:32.441183 25022 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:45:32.441193 25022 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38511/jwks.json
I20260619 02:45:32.442982 25022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:40337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:40981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38511/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:32.443207 25022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:32.443406 25022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:32.448272 25030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:32.448297 25028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:32.448297 25022 server_base.cc:1034] running on GCE node
W20260619 02:45:32.448320 25027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:32.448697 25022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:32.448987 25022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:32.450151 25022 hybrid_clock.cc:648] HybridClock initialized: now 1781837132450139 us; error 28 us; skew 500 ppm
I20260619 02:45:32.451484 25022 webserver.cc:460] Webserver started at http://127.19.113.126:36153/ using document root <none> and password file <none>
I20260619 02:45:32.451692 25022 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:32.451742 25022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:32.451860 25022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:32.452876 25022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "4f72d51b3a8e47d8949968ad47bfa7df"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "2e18a7affc589f01efedf5daf07fa472"
server_key_iv: "2eb55a1ee7f33c7a5b06eeb5f06a2066"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.453210 25022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "4f72d51b3a8e47d8949968ad47bfa7df"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "2e18a7affc589f01efedf5daf07fa472"
server_key_iv: "2eb55a1ee7f33c7a5b06eeb5f06a2066"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.454511 25022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:45:32.455346 25036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:32.455536 25022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:32.455623 25022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "4f72d51b3a8e47d8949968ad47bfa7df"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "2e18a7affc589f01efedf5daf07fa472"
server_key_iv: "2eb55a1ee7f33c7a5b06eeb5f06a2066"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.455685 25022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:32.469718 25022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:32.470053 25022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:32.470183 25022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:32.474087 25022 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:40981
I20260619 02:45:32.474121 25088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:40981 every 8 connection(s)
I20260619 02:45:32.474462 25022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:32.475464 25089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:32.477723 25089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df: Bootstrap starting.
I20260619 02:45:32.478261 25089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:32.478487 25089 log.cc:824] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:32.479167 25089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df: No bootstrap required, opened a new log
I20260619 02:45:32.480686 25089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f72d51b3a8e47d8949968ad47bfa7df" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40981 } }
I20260619 02:45:32.480823 25089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:32.480844 25089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f72d51b3a8e47d8949968ad47bfa7df, State: Initialized, Role: FOLLOWER
I20260619 02:45:32.480942 25089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f72d51b3a8e47d8949968ad47bfa7df" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40981 } }
I20260619 02:45:32.480991 25089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:32.481009 25089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:32.481045 25089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:32.481498 25089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f72d51b3a8e47d8949968ad47bfa7df" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40981 } }
I20260619 02:45:32.481580 25089 leader_election.cc:302] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f72d51b3a8e47d8949968ad47bfa7df; no voters:
I20260619 02:45:32.481698 25089 leader_election.cc:288] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:32.481756 25092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:32.481911 25092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [term 1 LEADER]: Becoming Leader. State: Replica: 4f72d51b3a8e47d8949968ad47bfa7df, State: Running, Role: LEADER
I20260619 02:45:32.481921 25089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:32.482020 25092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f72d51b3a8e47d8949968ad47bfa7df" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40981 } }
I20260619 02:45:32.482359 25094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f72d51b3a8e47d8949968ad47bfa7df. Latest consensus state: current_term: 1 leader_uuid: "4f72d51b3a8e47d8949968ad47bfa7df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f72d51b3a8e47d8949968ad47bfa7df" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40981 } } }
I20260619 02:45:32.482452 25094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:32.482549 25093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f72d51b3a8e47d8949968ad47bfa7df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f72d51b3a8e47d8949968ad47bfa7df" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40981 } } }
I20260619 02:45:32.482615 25093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:32.482723 25101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:32.483327 25101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:32.483852 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25022
I20260619 02:45:32.483950 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:32.484129 19909 external_mini_cluster.cc:1428] Setting key 04328d85d672b52bc5c7dff0da558e58
I20260619 02:45:32.485328 25101 catalog_manager.cc:1347] Generated new cluster ID: 6c02cf82d5524fcebeb74fe22f40da1e
I20260619 02:45:32.485386 25101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:32.489168 25111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.484565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.19.113.126:40981)
0619 02:45:32.484786 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.484795 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.485000 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:45:32.485131 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.485139 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.485211 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.485346 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.485641 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.486532 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.487350 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.487796 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.487915 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.488557 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.488562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.488562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.488725 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.488756 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.488767 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.488821 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.488890 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.488966 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260619 02:45:32.490001 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:32.492604 25101 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:32.493256 25101 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:32.505193 25101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f72d51b3a8e47d8949968ad47bfa7df: Generated new TSK 0
I20260619 02:45:32.505416 25101 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:45:32.537424 25119 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 02:45:32.537533 25119 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:45:32.537691 25111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.515337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.19.113.126:40981)
0619 02:45:32.515388 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.515389 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.515544 (+ 155us) server_negotiation.cc:407] Connection header received
0619 02:45:32.515624 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.515625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.515639 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.515690 (+ 51us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:45:32.515844 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.516343 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.517094 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.517273 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.517330 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.517537 (+ 207us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:45:32.537454 (+ 19917us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:45:32.537621 (+ 167us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36590: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 02:45:32.537753 25111 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36590: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 02:45:32.538182 25118 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:32.515279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40981 (local address 127.0.0.1:36590)
0619 02:45:32.515452 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.515461 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.515527 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.515707 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.515708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.515710 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:45:32.515786 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:32.515788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.516420 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.516421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:32.517007 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:32.517009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.517349 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.517350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:32.517413 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.517432 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:45:32.537733 (+ 20301us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:45:32.537849 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.113.126:40981: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.113.126:40981: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 02:45:32.538642 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25022
2026-06-19T02:45:32Z chronyd exiting
I20260619 02:45:32.574189 19909 test_util.cc:175] -----------------------------------------------
I20260619 02:45:32.574275 19909 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837111403550-19909-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 02:45:32.575091 19909 mini_oidc.cc:152] Starting JWKS server
I20260619 02:45:32.575814 19909 webserver.cc:460] Webserver started at https://127.0.0.1:41329/ using document root <none> and password file <none>
I20260619 02:45:32.575865 19909 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:45:32.575995 19909 webserver.cc:460] Webserver started at http://127.0.0.1:34579/ using document root <none> and password file <none>
2026-06-19T02:45:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:32Z Disabled control of system clock
I20260619 02:45:32.583439 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:39281
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:39281
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41329/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/testchainca.cert with env {}
W20260619 02:45:32.663391 25125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:32.663606 25125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:32.663626 25125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:32.664752 25125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:32.664784 25125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:32.664795 25125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:32.664806 25125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:32.664817 25125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:45:32.664827 25125 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:45:32.664837 25125 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41329/jwks.json
I20260619 02:45:32.666308 25125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:39281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:39281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41329/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:32.666508 25125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:32.666679 25125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:32.671612 25131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:32.671693 25125 server_base.cc:1034] running on GCE node
W20260619 02:45:32.671593 25130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:32.671654 25133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:32.672083 25125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:32.672344 25125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:32.673511 25125 hybrid_clock.cc:648] HybridClock initialized: now 1781837132673495 us; error 29 us; skew 500 ppm
I20260619 02:45:32.674835 25125 webserver.cc:460] Webserver started at http://127.19.113.126:36319/ using document root <none> and password file <none>
I20260619 02:45:32.675100 25125 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:32.675153 25125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:32.675278 25125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:32.676409 25125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "d29b13603d6a4940a0f202b441337a07"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "e4beb5dab61c7e116ed385e96481553e"
server_key_iv: "fac7a545f0171021f0abd1a6b347d741"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.676787 25125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "d29b13603d6a4940a0f202b441337a07"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "e4beb5dab61c7e116ed385e96481553e"
server_key_iv: "fac7a545f0171021f0abd1a6b347d741"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.678045 25125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:45:32.678809 25139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:32.679003 25125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:32.679085 25125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "d29b13603d6a4940a0f202b441337a07"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "e4beb5dab61c7e116ed385e96481553e"
server_key_iv: "fac7a545f0171021f0abd1a6b347d741"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.679157 25125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:32.688923 25125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:32.689260 25125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:32.689388 25125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:32.693696 25125 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:39281
I20260619 02:45:32.693743 25191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:39281 every 8 connection(s)
I20260619 02:45:32.694140 25125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:32.694933 25192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:32.697489 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25125
I20260619 02:45:32.697597 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:32.697757 19909 external_mini_cluster.cc:1428] Setting key ce949ff09c36543b44f9afc34eab7f14
I20260619 02:45:32.698344 25192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07: Bootstrap starting.
I20260619 02:45:32.699023 25192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:32.699334 25192 log.cc:824] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:32.703572 25192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07: No bootstrap required, opened a new log
I20260619 02:45:32.705891 25192 raft_consensus.cc:348] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b13603d6a4940a0f202b441337a07" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39281 } }
I20260619 02:45:32.706089 25192 raft_consensus.cc:374] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:32.706139 25192 raft_consensus.cc:729] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d29b13603d6a4940a0f202b441337a07, State: Initialized, Role: FOLLOWER
I20260619 02:45:32.706276 25192 consensus_queue.cc:260] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b13603d6a4940a0f202b441337a07" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39281 } }
I20260619 02:45:32.706346 25192 raft_consensus.cc:388] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:32.706380 25192 raft_consensus.cc:482] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:32.706429 25192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:32.707202 25192 raft_consensus.cc:504] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b13603d6a4940a0f202b441337a07" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39281 } }
I20260619 02:45:32.707304 25192 leader_election.cc:302] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d29b13603d6a4940a0f202b441337a07; no voters:
I20260619 02:45:32.707469 25192 leader_election.cc:288] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:32.707505 25197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:32.707675 25197 raft_consensus.cc:686] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [term 1 LEADER]: Becoming Leader. State: Replica: d29b13603d6a4940a0f202b441337a07, State: Running, Role: LEADER
I20260619 02:45:32.707720 25192 sys_catalog.cc:564] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:32.707760 25197 consensus_queue.cc:237] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b13603d6a4940a0f202b441337a07" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39281 } }
I20260619 02:45:32.708078 25197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d29b13603d6a4940a0f202b441337a07. Latest consensus state: current_term: 1 leader_uuid: "d29b13603d6a4940a0f202b441337a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b13603d6a4940a0f202b441337a07" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39281 } } }
I20260619 02:45:32.708139 25197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:32.708418 25198 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d29b13603d6a4940a0f202b441337a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29b13603d6a4940a0f202b441337a07" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39281 } } }
I20260619 02:45:32.708488 25198 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:32.709570 25202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:32.710143 25202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:32.711680 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.698281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.19.113.126:39281)
0619 02:45:32.698542 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.698553 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.704845 (+ 6292us) server_negotiation.cc:407] Connection header received
0619 02:45:32.706154 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.706166 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.706259 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.706428 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.706648 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.708520 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.709429 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.709991 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.710110 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.710965 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.710969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.710969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.711167 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.711207 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.711221 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.711277 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.711394 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.711489 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260619 02:45:32.712026 25202 catalog_manager.cc:1347] Generated new cluster ID: d88882cb571f456eb958b139ad963693
I20260619 02:45:32.712080 25202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:32.712559 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:32.727830 25202 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:32.728343 25202 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:32.732748 25202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d29b13603d6a4940a0f202b441337a07: Generated new TSK 0
I20260619 02:45:32.732954 25202 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:45:32.762796 25222 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 02:45:32.762916 25222 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:45:32.763255 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.741130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60642 (local address 127.19.113.126:39281)
0619 02:45:32.741174 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.741176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.741250 (+ 74us) server_negotiation.cc:407] Connection header received
0619 02:45:32.741338 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.741338 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.741348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.741404 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:45:32.741535 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.742067 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.743184 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.743384 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.743436 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.743595 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:45:32.762998 (+ 19403us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:45:32.763182 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60642: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:45:32.763324 25195 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60642: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 02:45:32.763341 25221 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:32.741046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:39281 (local address 127.0.0.1:60642)
0619 02:45:32.741190 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.741202 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.741270 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.741417 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.741418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.741420 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:45:32.741479 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:32.741482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.742136 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.742136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:32.743103 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:32.743107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.743454 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.743455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:32.743513 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.743519 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:45:32.763136 (+ 19617us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:45:32.763249 (+ 113us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.113.126:39281: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260619 02:45:32.763698 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25125
2026-06-19T02:45:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 02:45:32.802362 19909 mini_oidc.cc:152] Starting JWKS server
I20260619 02:45:32.803087 19909 webserver.cc:460] Webserver started at https://127.0.0.1:34525/ using document root <none> and password file <none>
I20260619 02:45:32.803133 19909 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:45:32.803264 19909 webserver.cc:460] Webserver started at http://127.0.0.1:42071/ using document root <none> and password file <none>
2026-06-19T02:45:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:32Z Disabled control of system clock
I20260619 02:45:32.810357 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:34735
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:34735
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34525/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:45:32.890254 25228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:32.890465 25228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:32.890486 25228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:32.891714 25228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:32.891759 25228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:32.891770 25228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:32.891781 25228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:32.891793 25228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:45:32.891805 25228 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:45:32.891836 25228 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34525/jwks.json
I20260619 02:45:32.893217 25228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:34735
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:34735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34525/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:32.893460 25228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:32.893736 25228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:32.898422 25236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:32.898452 25234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:32.898483 25233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:32.898571 25228 server_base.cc:1034] running on GCE node
I20260619 02:45:32.898954 25228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:32.899232 25228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:32.900401 25228 hybrid_clock.cc:648] HybridClock initialized: now 1781837132900384 us; error 29 us; skew 500 ppm
I20260619 02:45:32.901566 25228 webserver.cc:460] Webserver started at http://127.19.113.126:46119/ using document root <none> and password file <none>
I20260619 02:45:32.901773 25228 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:32.901820 25228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:32.901937 25228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:32.903043 25228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "865a8633883a4e938db4650a46605f7f"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "1dfc975cdee46f2acc8ce9ab5b28ee37"
server_key_iv: "79b0beb179fa23194b7c38e288a0b714"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.903364 25228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "865a8633883a4e938db4650a46605f7f"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "1dfc975cdee46f2acc8ce9ab5b28ee37"
server_key_iv: "79b0beb179fa23194b7c38e288a0b714"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.904559 25228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:32.905309 25242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:32.905481 25228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:32.905562 25228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "865a8633883a4e938db4650a46605f7f"
format_stamp: "Formatted at 2026-06-19 02:45:32 on dist-test-slave-zqpb"
server_key: "1dfc975cdee46f2acc8ce9ab5b28ee37"
server_key_iv: "79b0beb179fa23194b7c38e288a0b714"
server_key_version: "encryptionkey@0"
I20260619 02:45:32.905623 25228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:32.918547 25228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:32.918836 25228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:32.919003 25228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:32.922600 25228 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:34735
I20260619 02:45:32.922632 25294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:34735 every 8 connection(s)
I20260619 02:45:32.922956 25228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:32.923804 25295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:32.924077 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25228
I20260619 02:45:32.924155 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:32.924311 19909 external_mini_cluster.cc:1428] Setting key 37d6bd76f4ce4500e6a6c3817102c41d
I20260619 02:45:32.926363 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f: Bootstrap starting.
I20260619 02:45:32.926972 25295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:32.927258 25295 log.cc:824] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:32.927932 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f: No bootstrap required, opened a new log
I20260619 02:45:32.929538 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.924794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.19.113.126:34735)
0619 02:45:32.924981 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.924988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.925014 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:45:32.925413 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.925419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.925476 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.925587 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:32.925813 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.926737 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.927750 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.928195 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.928283 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.928943 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:32.928947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:32.928947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:32.929100 (+ 153us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:32.929131 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:32.929141 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:32.929189 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:32.929278 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 02:45:32.929337 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260619 02:45:32.929976 25295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865a8633883a4e938db4650a46605f7f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34735 } }
I20260619 02:45:32.930137 25295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:32.930186 25295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 865a8633883a4e938db4650a46605f7f, State: Initialized, Role: FOLLOWER
I20260619 02:45:32.930301 25295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865a8633883a4e938db4650a46605f7f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34735 } }
I20260619 02:45:32.930367 25295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:32.930395 25295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:32.930445 25295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:32.930496 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:32.931001 25295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865a8633883a4e938db4650a46605f7f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34735 } }
I20260619 02:45:32.931092 25295 leader_election.cc:302] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 865a8633883a4e938db4650a46605f7f; no voters:
I20260619 02:45:32.931241 25295 leader_election.cc:288] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:32.931272 25300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:32.931373 25300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [term 1 LEADER]: Becoming Leader. State: Replica: 865a8633883a4e938db4650a46605f7f, State: Running, Role: LEADER
I20260619 02:45:32.931473 25295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:32.931457 25300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865a8633883a4e938db4650a46605f7f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34735 } }
I20260619 02:45:32.931763 25300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 865a8633883a4e938db4650a46605f7f. Latest consensus state: current_term: 1 leader_uuid: "865a8633883a4e938db4650a46605f7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865a8633883a4e938db4650a46605f7f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34735 } } }
I20260619 02:45:32.931784 25301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "865a8633883a4e938db4650a46605f7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865a8633883a4e938db4650a46605f7f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34735 } } }
I20260619 02:45:32.931831 25300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:32.931883 25301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:32.932022 25305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:32.932538 25305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:32.934034 25305 catalog_manager.cc:1347] Generated new cluster ID: 21f556da1cad4f3fa15bb9cade41c7bf
I20260619 02:45:32.934079 25305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:32.941054 25305 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:32.941576 25305 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:32.946417 25305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 865a8633883a4e938db4650a46605f7f: Generated new TSK 0
I20260619 02:45:32.946592 25305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:32.957412 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:32.953787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.19.113.126:34735)
0619 02:45:32.953833 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:32.953835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:32.953894 (+ 59us) server_negotiation.cc:407] Connection header received
0619 02:45:32.953953 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:32.953954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:32.953966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:32.954043 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:45:32.954130 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.954943 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.955705 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.956018 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.956085 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.956261 (+ 176us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:45:32.957200 (+ 939us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:45:32.957331 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51378: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 02:45:32.957471 25298 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51378: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 02:45:32.959278 25324 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:32.953713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:34735 (local address 127.0.0.1:51378)
0619 02:45:32.953834 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:32.953844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:32.953901 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:32.954041 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:32.954041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:32.954043 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:45:32.954092 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:32.954094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.955045 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.955046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:32.955636 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:32.955638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:32.956111 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:32.956112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:32.956162 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:32.956167 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:45:32.957294 (+ 1127us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:45:32.959183 (+ 1889us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.113.126:34735: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":1714,"thread_start_us":78,"threads_started":1}
I20260619 02:45:32.959583 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25228
E20260619 02:45:32.970117 25325 webserver.cc:533] Webserver: sslize failed:
E20260619 02:45:32.970197 25325 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T02:45:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (195 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T02:45:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:32Z Disabled control of system clock
I20260619 02:45:33.004973 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:39745
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:35925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:39745
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/testchainca.cert with env {}
W20260619 02:45:33.093811 25329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:33.094031 25329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:33.094053 25329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:33.095261 25329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:33.095304 25329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:33.095316 25329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:33.095327 25329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:33.095353 25329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:45:33.095369 25329 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:45:33.095383 25329 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 02:45:33.096870 25329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:39745
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:39745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:33.097097 25329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:33.097290 25329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:33.102000 25337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:33.102034 25335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:33.102010 25334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:33.102295 25329 server_base.cc:1034] running on GCE node
I20260619 02:45:33.102536 25329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:33.102843 25329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:33.104054 25329 hybrid_clock.cc:648] HybridClock initialized: now 1781837133104035 us; error 35 us; skew 500 ppm
I20260619 02:45:33.105279 25329 webserver.cc:460] Webserver started at http://127.19.113.126:38113/ using document root <none> and password file <none>
I20260619 02:45:33.105494 25329 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:33.105542 25329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:33.105664 25329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:33.106698 25329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "6cf49379785f481286f64764d913dbb6"
format_stamp: "Formatted at 2026-06-19 02:45:33 on dist-test-slave-zqpb"
server_key: "c3d14ae0ccdf4ceea71f7bd445165acb"
server_key_iv: "c2bdae40886da9111a0c46cbad62eaad"
server_key_version: "encryptionkey@0"
I20260619 02:45:33.107107 25329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "6cf49379785f481286f64764d913dbb6"
format_stamp: "Formatted at 2026-06-19 02:45:33 on dist-test-slave-zqpb"
server_key: "c3d14ae0ccdf4ceea71f7bd445165acb"
server_key_iv: "c2bdae40886da9111a0c46cbad62eaad"
server_key_version: "encryptionkey@0"
I20260619 02:45:33.108300 25329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:33.109015 25343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:33.109189 25329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:33.109268 25329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "6cf49379785f481286f64764d913dbb6"
format_stamp: "Formatted at 2026-06-19 02:45:33 on dist-test-slave-zqpb"
server_key: "c3d14ae0ccdf4ceea71f7bd445165acb"
server_key_iv: "c2bdae40886da9111a0c46cbad62eaad"
server_key_version: "encryptionkey@0"
I20260619 02:45:33.109333 25329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:33.116458 25329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:33.116743 25329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:33.116863 25329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:33.120551 25329 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:39745
I20260619 02:45:33.120626 25395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:39745 every 8 connection(s)
I20260619 02:45:33.120939 25329 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:33.121868 25396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:33.124169 25396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6: Bootstrap starting.
I20260619 02:45:33.124732 25396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:33.124997 25396 log.cc:824] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:33.125656 25396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6: No bootstrap required, opened a new log
I20260619 02:45:33.127331 25396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf49379785f481286f64764d913dbb6" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39745 } }
I20260619 02:45:33.127470 25396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:33.127490 25396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf49379785f481286f64764d913dbb6, State: Initialized, Role: FOLLOWER
I20260619 02:45:33.127624 25396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf49379785f481286f64764d913dbb6" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39745 } }
I20260619 02:45:33.127683 25396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:33.127705 25396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:33.127749 25396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:33.128265 25396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf49379785f481286f64764d913dbb6" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39745 } }
I20260619 02:45:33.128355 25396 leader_election.cc:302] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cf49379785f481286f64764d913dbb6; no voters:
I20260619 02:45:33.128511 25396 leader_election.cc:288] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:33.128556 25399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:33.128728 25399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [term 1 LEADER]: Becoming Leader. State: Replica: 6cf49379785f481286f64764d913dbb6, State: Running, Role: LEADER
I20260619 02:45:33.128756 25396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:33.128845 25399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf49379785f481286f64764d913dbb6" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39745 } }
I20260619 02:45:33.128991 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25329
I20260619 02:45:33.129084 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:33.129274 19909 external_mini_cluster.cc:1428] Setting key e9fb60cae6f566c48d3551fe6f3c70e1
I20260619 02:45:33.129559 25400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cf49379785f481286f64764d913dbb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf49379785f481286f64764d913dbb6" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39745 } } }
I20260619 02:45:33.129642 25400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:33.129766 25401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cf49379785f481286f64764d913dbb6. Latest consensus state: current_term: 1 leader_uuid: "6cf49379785f481286f64764d913dbb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf49379785f481286f64764d913dbb6" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39745 } } }
I20260619 02:45:33.129810 25401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:33.129837 25409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:33.130306 25409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:33.131847 25409 catalog_manager.cc:1347] Generated new cluster ID: 10f804fe68ce4f8fa530527e4b63c529
I20260619 02:45:33.131901 25409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:33.134577 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:33.130053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.19.113.126:39745)
0619 02:45:33.130262 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:33.130272 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:33.130320 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:45:33.130471 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:33.130479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:33.130554 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:33.130688 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:33.130890 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:33.132113 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:33.132850 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:33.133276 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:33.133382 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:33.133999 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:33.134003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:33.134004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:33.134165 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:33.134197 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:33.134207 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:33.134262 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:33.134330 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:45:33.134407 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260619 02:45:33.135406 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:33.152752 25409 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:33.153479 25409 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:33.162160 25409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cf49379785f481286f64764d913dbb6: Generated new TSK 0
I20260619 02:45:33.162356 25409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:33.239494 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:33.172960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.19.113.126:39745)
0619 02:45:33.173006 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:33.173008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:33.173071 (+ 63us) server_negotiation.cc:407] Connection header received
0619 02:45:33.173158 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:33.173159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:33.173169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:33.173225 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:45:33.173355 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:33.173815 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:33.174982 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:33.175166 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:33.175216 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:33.175376 (+ 160us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:45:33.239274 (+ 63898us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:45:33.239430 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45018: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 02:45:33.239562 25417 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45018: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 02:45:33.239704 25425 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:33.172869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:39745 (local address 127.0.0.1:45018)
0619 02:45:33.173012 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:33.173024 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:33.173090 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:33.173235 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:33.173236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:33.173238 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:45:33.173298 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:33.173301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:33.173897 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:33.173898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:33.174904 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:33.174909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:33.175234 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:33.175235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:33.175295 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:33.175300 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:45:33.239445 (+ 64145us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:45:33.239592 (+ 147us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.113.126:39745: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260619 02:45:33.240036 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25329
2026-06-19T02:45:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (259 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:33 dist-test-slave-zqpb krb5kdc[25437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:33 dist-test-slave-zqpb krb5kdc[25437](info): set up 2 sockets
Jun 19 02:45:33 dist-test-slave-zqpb krb5kdc[25437](info): commencing operation
krb5kdc: starting...
W20260619 02:45:34.358736 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:34 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:34.467744 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40613
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:34.547693 25450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:34.547923 25450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:34.547955 25450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:34.549144 25450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:34.549187 25450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:34.549206 25450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:34.549223 25450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:34.549239 25450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:34.550745 25450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:40613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:34.551026 25450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:34.551241 25450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:34.556015 25455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:34.556134 25450 server_base.cc:1034] running on GCE node
W20260619 02:45:34.556026 25458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:34.556026 25456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:34.556464 25450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:34.556756 25450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:34.557945 25450 hybrid_clock.cc:648] HybridClock initialized: now 1781837134557948 us; error 52 us; skew 500 ppm
Jun 19 02:45:34 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:34.566144 25450 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:34.566648 25450 webserver.cc:460] Webserver started at http://127.19.113.126:39417/ using document root <none> and password file <none>
I20260619 02:45:34.566923 25450 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:34.566983 25450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:34.567112 25450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:34.568183 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "2d04a78632c54ebaba0099f9480ee1da"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "a11e3d99080aa2ff866f041ea4b00165"
server_key_iv: "e24870ddf5914a7dde5cd0da9462a539"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.568535 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "2d04a78632c54ebaba0099f9480ee1da"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "a11e3d99080aa2ff866f041ea4b00165"
server_key_iv: "e24870ddf5914a7dde5cd0da9462a539"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.569872 25450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:34.570690 25465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:34.570993 25450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:34.571085 25450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "2d04a78632c54ebaba0099f9480ee1da"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "a11e3d99080aa2ff866f041ea4b00165"
server_key_iv: "e24870ddf5914a7dde5cd0da9462a539"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.571159 25450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:34.589677 25450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:34.591874 25450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:34.591990 25450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:34.595901 25450 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:40613
I20260619 02:45:34.595930 25517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:40613 every 8 connection(s)
I20260619 02:45:34.596280 25450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:34.597183 25518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:34.599503 25518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da: Bootstrap starting.
I20260619 02:45:34.600042 25518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:34.600296 25518 log.cc:824] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:34.600982 25518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da: No bootstrap required, opened a new log
I20260619 02:45:34.602447 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25450
I20260619 02:45:34.602538 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:34.602699 19909 external_mini_cluster.cc:1428] Setting key 8b3417b3222088d5ac452e348e9a2b4f
I20260619 02:45:34.602977 25518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04a78632c54ebaba0099f9480ee1da" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40613 } }
I20260619 02:45:34.603130 25518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:34.603165 25518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d04a78632c54ebaba0099f9480ee1da, State: Initialized, Role: FOLLOWER
I20260619 02:45:34.603289 25518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04a78632c54ebaba0099f9480ee1da" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40613 } }
I20260619 02:45:34.603367 25518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:34.603397 25518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:34.603448 25518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:34.604494 25518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04a78632c54ebaba0099f9480ee1da" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40613 } }
I20260619 02:45:34.604604 25518 leader_election.cc:302] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d04a78632c54ebaba0099f9480ee1da; no voters:
I20260619 02:45:34.604765 25518 leader_election.cc:288] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:34.604837 25523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:34.604969 25523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [term 1 LEADER]: Becoming Leader. State: Replica: 2d04a78632c54ebaba0099f9480ee1da, State: Running, Role: LEADER
I20260619 02:45:34.605043 25518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:34.605058 25523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04a78632c54ebaba0099f9480ee1da" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40613 } }
I20260619 02:45:34.605858 25523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d04a78632c54ebaba0099f9480ee1da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04a78632c54ebaba0099f9480ee1da" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40613 } } }
I20260619 02:45:34.605931 25523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:34.606055 25523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d04a78632c54ebaba0099f9480ee1da. Latest consensus state: current_term: 1 leader_uuid: "2d04a78632c54ebaba0099f9480ee1da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04a78632c54ebaba0099f9480ee1da" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40613 } } }
I20260619 02:45:34.606102 25523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da [sys.catalog]: This master's current role is: LEADER
W20260619 02:45:34.606794 25538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:34.606963 25538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:45:34.607034 25534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:34.607229 25534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:34.608742 25534 catalog_manager.cc:1347] Generated new cluster ID: b525d7bbbd5d4b28b632adebc811b109
I20260619 02:45:34.608791 25534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:34 dist-test-slave-zqpb krb5kdc[25437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:34.614166 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:34.603261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56278 (local address 127.19.113.126:40613)
0619 02:45:34.603466 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:34.603476 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:34.603510 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:45:34.603637 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:34.603644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:34.603700 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:34.603861 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:34.604524 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.605978 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.607079 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.607660 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.607769 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:34.610533 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:34.610539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:34.610539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:34.610552 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:34.612529 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:34.612903 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:34.612906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:34.612906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:34.613037 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:34.613210 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:34.613211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:34.613211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:34.613538 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:34.613713 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:34.613788 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:45:34.613940 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260619 02:45:34.634227 25534 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:34.634776 25534 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:34.637395 25534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d04a78632c54ebaba0099f9480ee1da: Generated new TSK 0
I20260619 02:45:34.637591 25534 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:34.665688 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40613
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:34.751724 25544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:34.751964 25544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:34.751991 25544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:34.753243 25544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:34.753293 25544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:34.753321 25544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:34.754935 25544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:34.755198 25544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:34.755422 25544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:34.760471 25549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:34.760535 25550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:34.760535 25552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:34.760604 25544 server_base.cc:1034] running on GCE node
I20260619 02:45:34.760990 25544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:34.761232 25544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:34.762409 25544 hybrid_clock.cc:648] HybridClock initialized: now 1781837134762389 us; error 29 us; skew 500 ppm
Jun 19 02:45:34 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:34.764962 25544 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:34.765451 25544 webserver.cc:460] Webserver started at http://127.19.113.65:44341/ using document root <none> and password file <none>
I20260619 02:45:34.765676 25544 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:34.765722 25544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:34.765848 25544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:34.767007 25544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "7023fd52412d4c758f95fa83b7c948ce"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "ebc13021b29a02a07730d3b96b005215"
server_key_iv: "e8b343a6630f066bac8c34fad299985a"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.767334 25544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "7023fd52412d4c758f95fa83b7c948ce"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "ebc13021b29a02a07730d3b96b005215"
server_key_iv: "e8b343a6630f066bac8c34fad299985a"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.768621 25544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:34.769400 25559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:34.769598 25544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:34.769686 25544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "7023fd52412d4c758f95fa83b7c948ce"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "ebc13021b29a02a07730d3b96b005215"
server_key_iv: "e8b343a6630f066bac8c34fad299985a"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.769748 25544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:34.798358 25544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:34.800601 25544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:34.800743 25544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:34.800971 25544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:34.801280 25544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:34.801321 25544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:34.801348 25544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:34.801366 25544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:34.807287 25544 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:46555
I20260619 02:45:34.807338 25672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:46555 every 8 connection(s)
I20260619 02:45:34.807650 25544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:34.810755 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25544
I20260619 02:45:34.810855 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:34.811064 19909 external_mini_cluster.cc:1428] Setting key c1eb1a0b98b0288a5d1af993412a783f
Jun 19 02:45:34 dist-test-slave-zqpb krb5kdc[25437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:34.817989 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:34.808475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:35521 (local address 127.19.113.126:40613)
0619 02:45:34.808527 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:34.808529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:34.808776 (+ 247us) server_negotiation.cc:407] Connection header received
0619 02:45:34.809829 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:34.809831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:34.809848 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:34.809904 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:34.810520 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.811140 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.812440 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.812642 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.812697 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:34.815214 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:34.815216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:34.815216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:34.815224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:34.816671 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:34.817092 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:34.817093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:34.817093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:34.817153 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:34.817374 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:34.817375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:34.817375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:34.817494 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:34.817548 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:34.817807 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 02:45:34.817898 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:34.818162 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:34.808490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40613 (local address 127.19.113.65:35521)
0619 02:45:34.808650 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:34.808691 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:34.809680 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:34.809977 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:34.809980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:34.810253 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:34.810452 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:34.810456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.811236 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.811237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:34.812349 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:34.812352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.812715 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.812716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:34.812859 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:34.813462 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:34.813470 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:34.815106 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:34.816764 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:34.816772 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:34.816775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:34.817021 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:34.817205 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:34.817206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:34.817206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:34.817315 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:34.817545 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:34.817551 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:34.817692 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 02:45:34.817780 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:45:34.817941 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
I20260619 02:45:34.818955 25673 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40613
I20260619 02:45:34.819058 25673 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:34.819239 25673 heartbeater.cc:511] Master 127.19.113.126:40613 requested a full tablet report, sending...
I20260619 02:45:34.819785 25482 ts_manager.cc:194] Registered new tserver with Master: 7023fd52412d4c758f95fa83b7c948ce (127.19.113.65:46555)
I20260619 02:45:34.820475 25482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:35521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:34.835513 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40613
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:34.916961 25678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:34.917187 25678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:34.917212 25678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:34.918416 25678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:34.918470 25678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:34.918498 25678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:34.920069 25678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:34.920306 25678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:34.920545 25678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:34.925813 25683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:34.925971 25684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:34.925908 25678 server_base.cc:1034] running on GCE node
W20260619 02:45:34.925825 25686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:34.926389 25678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:34.926664 25678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:34.927824 25678 hybrid_clock.cc:648] HybridClock initialized: now 1781837134927744 us; error 95 us; skew 500 ppm
Jun 19 02:45:34 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:34.930389 25678 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:34.930902 25678 webserver.cc:460] Webserver started at http://127.19.113.66:45789/ using document root <none> and password file <none>
I20260619 02:45:34.931131 25678 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:34.931183 25678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:34.931305 25678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:34.932368 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "034965a000204147a1bd7f3f91af3fcb"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "a820952d57843cee6ffded9c8086734e"
server_key_iv: "80f86e627410d31b61bbf3755ffbabe3"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.932703 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "034965a000204147a1bd7f3f91af3fcb"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "a820952d57843cee6ffded9c8086734e"
server_key_iv: "80f86e627410d31b61bbf3755ffbabe3"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.934207 25678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:34.935094 25693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:34.935262 25678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:34.935350 25678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "034965a000204147a1bd7f3f91af3fcb"
format_stamp: "Formatted at 2026-06-19 02:45:34 on dist-test-slave-zqpb"
server_key: "a820952d57843cee6ffded9c8086734e"
server_key_iv: "80f86e627410d31b61bbf3755ffbabe3"
server_key_version: "encryptionkey@0"
I20260619 02:45:34.935422 25678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:34.953851 25678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:34.955958 25678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:34.956097 25678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:34.956323 25678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:34.956625 25678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:34.956658 25678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:34.956687 25678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:34.956712 25678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:34.962422 25678 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:33845
I20260619 02:45:34.962476 25806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:33845 every 8 connection(s)
I20260619 02:45:34.962778 25678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:45:34 dist-test-slave-zqpb krb5kdc[25437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:34.970584 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25678
I20260619 02:45:34.970695 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:34.970913 19909 external_mini_cluster.cc:1428] Setting key 820abf077dae16c445d7c7b6aaac5964
I20260619 02:45:34.972024 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:34.963548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:59185 (local address 127.19.113.126:40613)
0619 02:45:34.963628 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:34.963630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:34.963772 (+ 142us) server_negotiation.cc:407] Connection header received
0619 02:45:34.964675 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:34.964676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:34.964687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:34.964748 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:34.965339 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.965875 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.966907 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.967106 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.967164 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:34.969213 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:34.969215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:34.969215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:34.969222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:34.970630 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:34.971098 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:34.971099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:34.971100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:34.971150 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:34.971401 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:34.971402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:34.971402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:34.971508 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:34.971562 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:34.971821 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 02:45:34.971907 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:45:34.972196 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:34.963488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40613 (local address 127.19.113.66:59185)
0619 02:45:34.963653 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:34.963696 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:34.964583 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:34.964804 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:34.964807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:34.965061 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:34.965255 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:34.965259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.965959 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.965960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:34.966792 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:34.966794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:34.967182 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:34.967183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:34.967299 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:34.967790 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:34.967798 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:34.969103 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:34.970721 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:34.970729 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:34.970732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:34.971028 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:34.971225 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:34.971226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:34.971227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:34.971338 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:34.971562 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:34.971569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:34.971706 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 02:45:34.971797 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:45:34.971973 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260619 02:45:34.972654 25807 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40613
I20260619 02:45:34.972750 25807 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:34.972935 25807 heartbeater.cc:511] Master 127.19.113.126:40613 requested a full tablet report, sending...
I20260619 02:45:34.973320 25482 ts_manager.cc:194] Registered new tserver with Master: 034965a000204147a1bd7f3f91af3fcb (127.19.113.66:33845)
I20260619 02:45:34.973763 25482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:59185
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:34.997617 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40613
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:35.080039 25812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:35.080265 25812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:35.080291 25812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:35.081460 25812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:35.081501 25812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:35.081529 25812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:35.083114 25812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:35.083359 25812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:35.083556 25812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:35.088886 25818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:35.088984 25812 server_base.cc:1034] running on GCE node
W20260619 02:45:35.088984 25820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:35.088912 25817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:35.089381 25812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:35.089643 25812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:35.090802 25812 hybrid_clock.cc:648] HybridClock initialized: now 1781837135090774 us; error 37 us; skew 500 ppm
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:35.093324 25812 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:35.093752 25812 webserver.cc:460] Webserver started at http://127.19.113.67:42377/ using document root <none> and password file <none>
I20260619 02:45:35.093952 25812 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:35.093990 25812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:35.094082 25812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:35.095178 25812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "f470f716028740d1a44589f44cd93bcd"
format_stamp: "Formatted at 2026-06-19 02:45:35 on dist-test-slave-zqpb"
server_key: "130aaf40f07331ce5e8106e87fb6595b"
server_key_iv: "6a28e69e686355d5e8dd4553ca7db048"
server_key_version: "encryptionkey@0"
I20260619 02:45:35.095481 25812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "f470f716028740d1a44589f44cd93bcd"
format_stamp: "Formatted at 2026-06-19 02:45:35 on dist-test-slave-zqpb"
server_key: "130aaf40f07331ce5e8106e87fb6595b"
server_key_iv: "6a28e69e686355d5e8dd4553ca7db048"
server_key_version: "encryptionkey@0"
I20260619 02:45:35.096985 25812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:35.097771 25827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:35.098003 25812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:35.098090 25812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "f470f716028740d1a44589f44cd93bcd"
format_stamp: "Formatted at 2026-06-19 02:45:35 on dist-test-slave-zqpb"
server_key: "130aaf40f07331ce5e8106e87fb6595b"
server_key_iv: "6a28e69e686355d5e8dd4553ca7db048"
server_key_version: "encryptionkey@0"
I20260619 02:45:35.098150 25812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:35.116343 25812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:35.118772 25812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:35.118995 25812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:35.119218 25812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:35.119496 25812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:35.119534 25812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:35.119557 25812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:35.119577 25812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:35.125417 25812 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:46677
I20260619 02:45:35.125505 25940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:46677 every 8 connection(s)
I20260619 02:45:35.125771 25812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:35.132772 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25812
I20260619 02:45:35.132843 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:35.133003 19909 external_mini_cluster.cc:1428] Setting key 3920856ada591be474ab2cc2559c7371
I20260619 02:45:35.135458 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:35.126566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:56191 (local address 127.19.113.126:40613)
0619 02:45:35.126651 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:35.126653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:35.126753 (+ 100us) server_negotiation.cc:407] Connection header received
0619 02:45:35.127738 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:35.127739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:35.127750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:35.127822 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:35.128497 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.129054 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.130158 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.130346 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.130408 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:35.132654 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:35.132655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:35.132656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:35.132663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:35.134055 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:35.134425 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:35.134426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:35.134427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:35.134483 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:35.134703 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:35.134704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:35.134705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:35.134815 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:35.134863 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:35.135300 (+ 437us) server_negotiation.cc:299] Negotiation successful
0619 02:45:35.135378 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:45:35.135597 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:35.126456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40613 (local address 127.19.113.67:56191)
0619 02:45:35.126623 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:35.126666 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:35.127621 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:35.127881 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:35.127884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:35.128181 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:35.128377 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:35.128382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.129155 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.129156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:35.130075 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:35.130078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.130475 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.130477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:35.130589 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:35.131127 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:35.131135 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:35.132546 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:35.134125 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:35.134131 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:35.134133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:35.134356 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:35.134549 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:35.134550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:35.134550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:35.134651 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:35.134893 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:35.134899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:35.135119 (+ 220us) client_negotiation.cc:769] Sending connection context
0619 02:45:35.135226 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 02:45:35.135399 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260619 02:45:35.136068 25941 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40613
I20260619 02:45:35.136152 25941 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:35.136327 25941 heartbeater.cc:511] Master 127.19.113.126:40613 requested a full tablet report, sending...
I20260619 02:45:35.136648 25482 ts_manager.cc:194] Registered new tserver with Master: f470f716028740d1a44589f44cd93bcd (127.19.113.67:46677)
I20260619 02:45:35.136998 25482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:56191
I20260619 02:45:35.144762 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/test-admin.keytab.
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:35.159893 19909 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 02:45:35.161499 25948 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:35.160787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.19.113.65:46555)
0619 02:45:35.160941 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:35.160946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:35.160954 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:35.160978 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:35.160979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:35.160991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:35.161052 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:35.161433 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35586: BlockingRecv error: recv got EOF from 127.0.0.1:35586 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
W20260619 02:45:35.161644 19909 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:35.162643 19909 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:35.163524 19909 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:35.171442 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:35.164046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35592 (local address 127.19.113.65:46555)
0619 02:45:35.164099 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:35.164100 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:35.164109 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:35.164175 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:35.164176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:35.164186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:35.164233 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:35.164571 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.165307 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.166052 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.166379 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.166459 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:35.168871 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:35.168880 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:35.168880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:35.168888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:35.170392 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:35.170723 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:35.170726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:35.170726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:35.170781 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:35.170962 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:35.170963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:35.170963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:35.171113 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:35.171225 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:35.171274 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 02:45:35.171335 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 02:45:35.173265 19909 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5cc' of type FILE
W20260619 02:45:35.173285 25948 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:35.172673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.19.113.65:46555)
0619 02:45:35.172723 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:35.172725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:35.172756 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:45:35.172807 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:35.172807 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:35.172816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:35.172905 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:35.173217 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35596: BlockingRecv error: recv got EOF from 127.0.0.1:35596 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:35.174229 19909 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:35.175156 19909 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:35.181380 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:35.175619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35598 (local address 127.19.113.65:46555)
0619 02:45:35.175668 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:35.175669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:35.175693 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:45:35.175744 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:35.175744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:35.175752 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:35.175793 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:35.176095 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.176579 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.177212 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.177400 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.177442 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:35.179263 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:35.179264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:35.179264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:35.179272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:35.180429 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:35.180725 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:35.180726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:35.180727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:35.180777 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:35.180936 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:35.180937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:35.180937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:35.181033 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:35.181115 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:35.181215 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:45:35.181291 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260619 02:45:35.182798 25948 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:35.182286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.19.113.65:46555)
0619 02:45:35.182321 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:35.182323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:35.182374 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:45:35.182413 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:35.182414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:35.182421 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:35.182460 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:35.182760 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35602: BlockingRecv error: recv got EOF from 127.0.0.1:35602 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:45:35.182895 19909 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837111403550-19909-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:35.183779 19909 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:35.184554 19909 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:45:35.191725 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:35.185105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.19.113.65:46555)
0619 02:45:35.185156 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:35.185158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:35.185166 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:35.185225 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:35.185226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:35.185234 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:35.185278 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:35.185619 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.186108 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.186761 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:35.187019 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:35.187076 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:35.189162 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:35.189163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:35.189164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:35.189172 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:35.190642 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:35.191011 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:35.191013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:35.191013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:35.191069 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:35.191229 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:35.191230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:35.191230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:35.191337 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:35.191422 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:35.191550 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 02:45:35.191624 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:45:35.192310 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25544
I20260619 02:45:35.196257 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25678
I20260619 02:45:35.199963 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25812
I20260619 02:45:35.203651 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25450
2026-06-19T02:45:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1793 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25961](info): set up 2 sockets
Jun 19 02:45:35 dist-test-slave-zqpb krb5kdc[25961](info): commencing operation
krb5kdc: starting...
W20260619 02:45:36.159013 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.126.keytab.
Entry for principal oryx/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.126.keytab.
I20260619 02:45:36.270398 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:32985
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.126.keytab
--principal=oryx/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:32985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:36.349968 25974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:36.350175 25974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:36.350205 25974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:36.351402 25974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:36.351446 25974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:36.351464 25974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:36.351481 25974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:36.351496 25974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:36.352989 25974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:32985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.126.keytab
--principal=oryx/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:32985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:36.353232 25974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:36.353452 25974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:45:36.358325 25974 server_base.cc:1034] running on GCE node
W20260619 02:45:36.358204 25980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:36.358232 25982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:36.358204 25979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:36.358827 25974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:36.359141 25974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:36.360287 25974 hybrid_clock.cc:648] HybridClock initialized: now 1781837136360273 us; error 28 us; skew 500 ppm
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:36.362847 25974 init.cc:377] Logged in from keytab as oryx/127.19.113.126@KRBTEST.COM (short username oryx)
I20260619 02:45:36.363307 25974 webserver.cc:460] Webserver started at http://127.19.113.126:34063/ using document root <none> and password file <none>
I20260619 02:45:36.363564 25974 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:36.363613 25974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:36.363744 25974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:36.364852 25974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "145a8ff47a8741ab8202851bc659808f"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "1002ebb48e6156a6edb422656e092469"
server_key_iv: "76bbc8a9aa424fe07c7469c1317b64e8"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.365178 25974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "145a8ff47a8741ab8202851bc659808f"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "1002ebb48e6156a6edb422656e092469"
server_key_iv: "76bbc8a9aa424fe07c7469c1317b64e8"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.366523 25974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:36.367345 25989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.367504 25974 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.367587 25974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "145a8ff47a8741ab8202851bc659808f"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "1002ebb48e6156a6edb422656e092469"
server_key_iv: "76bbc8a9aa424fe07c7469c1317b64e8"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.367650 25974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:36.382774 25974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:36.384836 25974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:36.384985 25974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:36.388723 25974 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:32985
I20260619 02:45:36.388767 26041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:32985 every 8 connection(s)
I20260619 02:45:36.389091 25974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:36.390080 26042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:36.392310 26042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: Bootstrap starting.
I20260619 02:45:36.392861 26042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:36.393113 26042 log.cc:824] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:36.393767 26042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: No bootstrap required, opened a new log
I20260619 02:45:36.395092 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 25974
I20260619 02:45:36.395183 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:36.395352 19909 external_mini_cluster.cc:1428] Setting key 3a28c19ea44b7c8cc79e084f44230e43
I20260619 02:45:36.395484 26042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } }
I20260619 02:45:36.395620 26042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:36.395653 26042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145a8ff47a8741ab8202851bc659808f, State: Initialized, Role: FOLLOWER
I20260619 02:45:36.395761 26042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } }
I20260619 02:45:36.395838 26042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:36.395874 26042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:36.395915 26042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:36.396893 26042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } }
I20260619 02:45:36.397003 26042 leader_election.cc:302] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 145a8ff47a8741ab8202851bc659808f; no voters:
I20260619 02:45:36.397148 26042 leader_election.cc:288] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:36.397186 26047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:36.397320 26047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 1 LEADER]: Becoming Leader. State: Replica: 145a8ff47a8741ab8202851bc659808f, State: Running, Role: LEADER
I20260619 02:45:36.397396 26042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:36.397403 26047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } }
I20260619 02:45:36.397951 26048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 145a8ff47a8741ab8202851bc659808f. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } }
I20260619 02:45:36.398090 26048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:36.398274 26049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } }
I20260619 02:45:36.398388 26049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:36.398470 26056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:36.399154 26056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:36.401794 26056 catalog_manager.cc:1347] Generated new cluster ID: 8a5f132352ea4bffa4599117c6d37317
I20260619 02:45:36.401852 26056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:36.406220 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:36.395806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40762 (local address 127.19.113.126:32985)
0619 02:45:36.395997 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:36.396005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:36.396036 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:45:36.396137 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:36.396143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:36.396191 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:36.396298 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:36.397065 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.398448 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.399477 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.399980 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.400080 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:36.402527 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:36.402532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:36.402532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:36.402544 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:36.404565 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:36.404956 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:36.404960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:36.404960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:36.405104 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:36.405277 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:36.405278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:36.405279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:36.405604 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:36.405768 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:36.405857 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 02:45:36.406006 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260619 02:45:36.408528 26056 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:36.409224 26056 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:36.419214 26056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: Generated new TSK 0
I20260619 02:45:36.419445 26056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab.
Entry for principal oryx/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab.
I20260619 02:45:36.445024 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32985
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab
--principal=oryx/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:36.525959 26068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:36.526158 26068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:36.526181 26068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:36.527393 26068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:36.527438 26068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:36.527464 26068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:36.528916 26068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab
--principal=oryx/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:36.529134 26068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:36.529358 26068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:36.534521 26076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:36.534435 26073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:36.534435 26074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:36.534641 26068 server_base.cc:1034] running on GCE node
I20260619 02:45:36.534852 26068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:36.535111 26068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:36.536285 26068 hybrid_clock.cc:648] HybridClock initialized: now 1781837136536271 us; error 29 us; skew 500 ppm
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:36.538738 26068 init.cc:377] Logged in from keytab as oryx/127.19.113.65@KRBTEST.COM (short username oryx)
I20260619 02:45:36.539232 26068 webserver.cc:460] Webserver started at http://127.19.113.65:40371/ using document root <none> and password file <none>
I20260619 02:45:36.539436 26068 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:36.539484 26068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:36.539608 26068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:36.540639 26068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "579681295fbc400ea66bae3b061cbfa7"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "37a5acfd1c66f5149a8dbaaa8874eb95"
server_key_iv: "7da40fd37c084aab262dc2d15572301f"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.540968 26068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "579681295fbc400ea66bae3b061cbfa7"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "37a5acfd1c66f5149a8dbaaa8874eb95"
server_key_iv: "7da40fd37c084aab262dc2d15572301f"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.542200 26068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:45:36.542936 26083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.543073 26068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:36.543154 26068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "579681295fbc400ea66bae3b061cbfa7"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "37a5acfd1c66f5149a8dbaaa8874eb95"
server_key_iv: "7da40fd37c084aab262dc2d15572301f"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.543219 26068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:36.551630 26068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:36.553581 26068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:36.553709 26068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:36.553915 26068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:36.554210 26068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:36.554243 26068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.554272 26068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:36.554296 26068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.559875 26068 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:43431
I20260619 02:45:36.559928 26196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:43431 every 8 connection(s)
I20260619 02:45:36.560233 26068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:36.569054 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:36.560898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:46109 (local address 127.19.113.126:32985)
0619 02:45:36.560953 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:36.560955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:36.561122 (+ 167us) server_negotiation.cc:407] Connection header received
0619 02:45:36.561640 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:36.561641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:36.561658 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:36.561709 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:36.562640 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.563276 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.564245 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.564459 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.564515 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:36.566559 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:36.566561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:36.566562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:36.566569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:36.567841 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:36.568201 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:36.568203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:36.568203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:36.568254 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:36.568466 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:36.568467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:36.568467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:36.568574 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:36.568628 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:36.568878 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 02:45:36.568963 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:45:36.569276 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:36.560858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:46109)
0619 02:45:36.560999 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:36.561040 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:36.561534 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:36.561777 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:36.561781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:36.562340 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:36.562536 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:36.562540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.563362 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.563363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:36.564152 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:36.564155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.564531 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.564532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:36.564648 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:36.565132 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:36.565140 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:36.566466 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:36.567921 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:36.567927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:36.567930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:36.568138 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:36.568306 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:36.568307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:36.568308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:36.568413 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:36.568627 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:36.568634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:36.568766 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:45:36.568850 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:45:36.569069 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260619 02:45:36.569761 26197 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32985
I20260619 02:45:36.569860 26197 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:36.570013 26197 heartbeater.cc:511] Master 127.19.113.126:32985 requested a full tablet report, sending...
I20260619 02:45:36.570248 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 26068
I20260619 02:45:36.570334 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:36.570580 19909 external_mini_cluster.cc:1428] Setting key 1d8f86d7364cdf3eb0a79080a25ec1bf
I20260619 02:45:36.570571 26006 ts_manager.cc:194] Registered new tserver with Master: 579681295fbc400ea66bae3b061cbfa7 (127.19.113.65:43431)
I20260619 02:45:36.571301 26006 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:46109
WARNING: no policy specified for oryx/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab.
Entry for principal oryx/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab.
I20260619 02:45:36.593796 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32985
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab
--principal=oryx/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:36.673527 26202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:36.673741 26202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:36.673772 26202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:36.674996 26202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:36.675040 26202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:36.675076 26202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:36.676553 26202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab
--principal=oryx/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:36.676785 26202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:36.677001 26202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:36.682158 26207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:36.682178 26208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:36.682292 26202 server_base.cc:1034] running on GCE node
W20260619 02:45:36.682160 26210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:36.682575 26202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:36.682821 26202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:36.684046 26202 hybrid_clock.cc:648] HybridClock initialized: now 1781837136684031 us; error 30 us; skew 500 ppm
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:36.686295 26202 init.cc:377] Logged in from keytab as oryx/127.19.113.66@KRBTEST.COM (short username oryx)
I20260619 02:45:36.686684 26202 webserver.cc:460] Webserver started at http://127.19.113.66:38955/ using document root <none> and password file <none>
I20260619 02:45:36.686888 26202 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:36.686949 26202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:36.687047 26202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:36.688069 26202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "ec3dcb4e39e0428c8ca53257cf94b645"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "eb6c435a6997b465c8274515237033bf"
server_key_iv: "b8c108792c4a3a9158c9c5600abfb53c"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.688400 26202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "ec3dcb4e39e0428c8ca53257cf94b645"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "eb6c435a6997b465c8274515237033bf"
server_key_iv: "b8c108792c4a3a9158c9c5600abfb53c"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.689594 26202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 02:45:36.690335 26217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.690523 26202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:36.690609 26202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "ec3dcb4e39e0428c8ca53257cf94b645"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "eb6c435a6997b465c8274515237033bf"
server_key_iv: "b8c108792c4a3a9158c9c5600abfb53c"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.690665 26202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:36.703863 26202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:36.705955 26202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:36.706063 26202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:36.706251 26202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:36.706549 26202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:36.706581 26202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.706604 26202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:36.706626 26202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.712224 26202 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:42895
I20260619 02:45:36.712263 26330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:42895 every 8 connection(s)
I20260619 02:45:36.712584 26202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:36.718775 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 26202
I20260619 02:45:36.718889 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:36.719076 19909 external_mini_cluster.cc:1428] Setting key c146697043bd9e4fe20d6f3f095a1995
I20260619 02:45:36.721726 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:36.713329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:38899 (local address 127.19.113.126:32985)
0619 02:45:36.713374 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:36.713376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:36.713604 (+ 228us) server_negotiation.cc:407] Connection header received
0619 02:45:36.714128 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:36.714129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:36.714141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:36.714188 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:36.715169 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.715686 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.716662 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.716857 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.716923 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:36.719010 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:36.719011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:36.719011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:36.719020 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:36.720436 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:36.720829 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:36.720830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:36.720830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:36.720889 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:36.721113 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:36.721115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:36.721115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:36.721236 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:36.721295 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:36.721545 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 02:45:36.721635 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:36.721972 26333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:36.713287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.66:38899)
0619 02:45:36.713474 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:36.713509 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:36.714028 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:36.714268 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:36.714272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:36.714835 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:36.715081 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:36.715087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.715771 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.715772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:36.716577 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:36.716579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.716927 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.716927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:36.717047 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:36.717550 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:36.717559 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:36.718848 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:36.720517 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:36.720523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:36.720525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:36.720764 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:36.720948 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:36.720948 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:36.720949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:36.721042 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:36.721295 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:36.721301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:36.721427 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 02:45:36.721514 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:45:36.721704 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260619 02:45:36.722416 26331 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32985
I20260619 02:45:36.722515 26331 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:36.722700 26331 heartbeater.cc:511] Master 127.19.113.126:32985 requested a full tablet report, sending...
I20260619 02:45:36.723101 26006 ts_manager.cc:194] Registered new tserver with Master: ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895)
I20260619 02:45:36.723464 26006 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:38899
WARNING: no policy specified for oryx/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab.
Entry for principal oryx/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab.
I20260619 02:45:36.742959 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32985
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab
--principal=oryx/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:36.823128 26336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:36.823338 26336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:36.823360 26336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:36.824508 26336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:36.824544 26336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:36.824574 26336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:36.825968 26336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab
--principal=oryx/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:36.826169 26336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:36.826352 26336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:36.831220 26341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:36.831368 26336 server_base.cc:1034] running on GCE node
W20260619 02:45:36.831259 26342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:36.831223 26344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:36.831687 26336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:36.831928 26336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:36.833081 26336 hybrid_clock.cc:648] HybridClock initialized: now 1781837136833059 us; error 30 us; skew 500 ppm
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:36.835515 26336 init.cc:377] Logged in from keytab as oryx/127.19.113.67@KRBTEST.COM (short username oryx)
I20260619 02:45:36.835909 26336 webserver.cc:460] Webserver started at http://127.19.113.67:39621/ using document root <none> and password file <none>
I20260619 02:45:36.836081 26336 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:36.836118 26336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:36.836207 26336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:36.837189 26336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "cc9e42730e6a05df898dfc61f8663df6"
server_key_iv: "b3877622cd8430394bb85e5a76b1d9d1"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.837468 26336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "cc9e42730e6a05df898dfc61f8663df6"
server_key_iv: "b3877622cd8430394bb85e5a76b1d9d1"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.838711 26336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:45:36.839486 26351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.839615 26336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.839691 26336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "cc9e42730e6a05df898dfc61f8663df6"
server_key_iv: "b3877622cd8430394bb85e5a76b1d9d1"
server_key_version: "encryptionkey@0"
I20260619 02:45:36.839756 26336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:36.858234 26336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:36.860651 26336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:36.860808 26336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:36.861033 26336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:36.861325 26336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:36.861356 26336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.861387 26336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:36.861403 26336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:36.866930 26336 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:40713
I20260619 02:45:36.866976 26464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:40713 every 8 connection(s)
I20260619 02:45:36.867300 26336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:36.867535 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 26336
I20260619 02:45:36.867645 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:36.867920 19909 external_mini_cluster.cc:1428] Setting key e6b4685924402ff5a3a7d64bd24c17dc
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:36.877175 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:36.868016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:34653 (local address 127.19.113.126:32985)
0619 02:45:36.868091 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:36.868094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:36.868341 (+ 247us) server_negotiation.cc:407] Connection header received
0619 02:45:36.868866 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:36.868867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:36.868884 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:36.868933 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:36.869972 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.870669 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.872155 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.872400 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.872464 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:36.874560 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:36.874561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:36.874562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:36.874569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:36.875927 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:36.876299 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:36.876301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:36.876301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:36.876353 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:36.876571 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:36.876572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:36.876572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:36.876682 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:36.876739 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:36.877005 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 02:45:36.877084 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:45:36.877360 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:36.868001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:34653)
0619 02:45:36.868238 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:36.868269 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:36.868771 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:36.868988 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:36.868991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:36.869566 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:36.869841 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:36.869849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.870757 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.870758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:36.872055 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:36.872058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:36.872480 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:36.872481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:36.872611 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:36.873114 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:36.873122 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:36.874470 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:36.876003 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:36.876010 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:36.876012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:36.876233 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:36.876406 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:36.876407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:36.876407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:36.876517 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:36.876736 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:36.876742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:36.876886 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:45:36.876974 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:45:36.877130 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
I20260619 02:45:36.877907 26465 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32985
I20260619 02:45:36.878007 26465 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:36.878176 26465 heartbeater.cc:511] Master 127.19.113.126:32985 requested a full tablet report, sending...
I20260619 02:45:36.878566 26006 ts_manager.cc:194] Registered new tserver with Master: dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713)
I20260619 02:45:36.878970 26006 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:34653
I20260619 02:45:36.879592 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.113.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.125.keytab.
Entry for principal oryx/127.19.113.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.125.keytab.
I20260619 02:45:36.903492 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.125:44391
--webserver_interface=127.19.113.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.125.keytab
--principal=oryx/127.19.113.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:32985,127.19.113.125:44391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:36.987546 26470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:36.987759 26470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:36.987780 26470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:36.989008 26470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:36.989053 26470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:36.989065 26470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:36.989078 26470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:36.989101 26470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:36.990605 26470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:32985,127.19.113.125:44391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.125.keytab
--principal=oryx/127.19.113.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.125:44391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.113.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
W20260619 02:45:36.990700 26470 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.113.126:32985,127.19.113.125:44391'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 02:45:36.990839 26470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:36.991086 26470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:36.995582 26476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:36.995586 26478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:36.995744 26470 server_base.cc:1034] running on GCE node
W20260619 02:45:36.995703 26475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:36.996078 26470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:36.996373 26470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:36.997553 26470 hybrid_clock.cc:648] HybridClock initialized: now 1781837136997535 us; error 34 us; skew 500 ppm
Jun 19 02:45:36 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:37.000294 26470 init.cc:377] Logged in from keytab as oryx/127.19.113.125@KRBTEST.COM (short username oryx)
I20260619 02:45:37.000744 26470 webserver.cc:460] Webserver started at http://127.19.113.125:36765/ using document root <none> and password file <none>
I20260619 02:45:37.000970 26470 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.001019 26470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.001142 26470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:37.002233 26470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data/instance:
uuid: "2d98ef611cf8467d98094879c3892ab4"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "563acf88defdfd431b625316dabc49f1"
server_key_iv: "33d19b62e15099f3d99a927e2bc421a3"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.002566 26470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal/instance:
uuid: "2d98ef611cf8467d98094879c3892ab4"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "563acf88defdfd431b625316dabc49f1"
server_key_iv: "33d19b62e15099f3d99a927e2bc421a3"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.003916 26470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:37.004582 26485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.004730 26470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:37.004809 26470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
uuid: "2d98ef611cf8467d98094879c3892ab4"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "563acf88defdfd431b625316dabc49f1"
server_key_iv: "33d19b62e15099f3d99a927e2bc421a3"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.004868 26470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.016898 26470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:37.019196 26470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:37.019335 26470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:37.023084 26470 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.125:44391
I20260619 02:45:37.023135 26537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.125:44391 every 8 connection(s)
I20260619 02:45:37.023438 26470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
I20260619 02:45:37.024386 26538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:37.025875 26538 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:37.028250 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 26470
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.125@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.113.125@KRBTEST.COM
I20260619 02:45:37.035029 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.026618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.19.113.126:32985)
0619 02:45:37.026689 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.026691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.026847 (+ 156us) server_negotiation.cc:407] Connection header received
0619 02:45:37.027356 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.027358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.027371 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.027423 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.028024 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.028582 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.029771 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.030125 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.030195 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.032376 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.032378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.032378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.032387 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.033722 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.034096 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.034097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.034098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.034149 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.034372 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.034373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.034373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.034518 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.034574 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.034823 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.034931 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:45:37.035279 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.026554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:40776)
0619 02:45:37.026736 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.026769 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.027260 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.027504 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.027507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.027745 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.027936 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.027940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.028807 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.028808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.029690 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.029693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.030207 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.030208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.030310 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.030802 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.030810 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.032286 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.033799 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.033805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.033807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.034028 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.034215 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.034216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.034217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.034316 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.034577 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.034583 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.034708 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.034795 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.035068 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260619 02:45:37.035876 26538 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:37.036288 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.028690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.19.113.125:44391)
0619 02:45:37.028842 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.028847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.028866 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:45:37.028883 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.028884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.028899 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.028958 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.029487 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.030367 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.031083 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.031370 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.031427 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.033591 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.033598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.033599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.033605 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.035079 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.035440 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.035443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.035443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.035498 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.035687 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.035688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.035688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.035856 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.035998 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.036069 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.036186 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.125@KRBTEST.COM for oryx/127.19.113.125@KRBTEST.COM
I20260619 02:45:37.042057 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.036125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.0.0.1:32798)
0619 02:45:37.036282 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.036291 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.036328 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.036454 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.036455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.036579 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.036611 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.036612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.037291 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.037292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.037786 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.037787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.038171 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.038172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.038218 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.038547 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.038550 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.039611 (+ 1061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.041317 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.041318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.041319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.041469 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.041606 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.041606 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.041607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.041651 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.041810 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.041811 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.041841 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.041886 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.041973 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260619 02:45:37.042140 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.036168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.19.113.125:44391)
0619 02:45:37.036310 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.036312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.036319 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:37.036386 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.036387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.036395 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.036451 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.036676 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.037214 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.037874 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.038102 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.038163 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.039677 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.039678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.039678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.039684 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.041253 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.041522 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.041523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.041523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.041566 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.041683 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.041683 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.041684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.041770 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.041811 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.041907 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.042018 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260619 02:45:37.043453 26538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Bootstrap starting.
I20260619 02:45:37.044070 26538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:37.044317 26538 log.cc:824] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:37.045087 26538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: No bootstrap required, opened a new log
I20260619 02:45:37.046826 26538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } }
I20260619 02:45:37.047024 26538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:37.047066 26538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d98ef611cf8467d98094879c3892ab4, State: Initialized, Role: FOLLOWER
I20260619 02:45:37.047174 26538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } }
I20260619 02:45:37.047452 26538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:37.047421 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } } }
I20260619 02:45:37.047526 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:45:37.049001 26556 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:37.049218 26556 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:45:37.050770 26470 master_runner.cc:428] Detected that this master 2d98ef611cf8467d98094879c3892ab4 is joining an existing cluster
I20260619 02:45:37.050817 26470 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.113.125:44391
I20260619 02:45:37.058115 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.051860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40784 (local address 127.19.113.126:32985)
0619 02:45:37.051902 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.051904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.052043 (+ 139us) server_negotiation.cc:407] Connection header received
0619 02:45:37.052145 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.052146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.052158 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.052188 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.052745 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.053239 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.054159 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.054445 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.054483 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.055569 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.055570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.055570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.055578 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.056984 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.057406 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.057407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.057407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.057463 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.057621 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.057622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.057622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.057732 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.057784 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.057934 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.058018 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:37.058140 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.051823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:40784)
0619 02:45:37.051986 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.051995 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.052069 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.052251 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.052251 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.052542 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.052653 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.052655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.053309 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.053310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.054067 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.054069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.054573 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.054573 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.054623 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.055106 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.055111 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.055490 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.057069 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.057070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.057071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.057336 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.057516 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.057516 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.057517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.057572 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.057782 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.057783 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.057831 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.057902 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.058045 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":12,"thread_start_us":89,"threads_started":1}
I20260619 02:45:37.059039 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.051855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.19.113.125:44391)
0619 02:45:37.051902 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.051904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.052139 (+ 235us) server_negotiation.cc:407] Connection header received
0619 02:45:37.052184 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.052184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.052194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.052252 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.052726 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.053527 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.054466 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.055188 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.055261 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.056133 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.056135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.056135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.056142 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.057738 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.058123 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.058124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.058124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.058171 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.058548 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.058549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.058549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.058653 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.058688 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.058854 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.058953 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:37.059422 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.051950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.0.0.1:32802)
0619 02:45:37.052068 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.052077 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.052135 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.052251 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.052252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.052538 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.052653 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.052655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.053614 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.053615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.054389 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.054391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.055270 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.055272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.055337 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.055736 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.055740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.056054 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.057819 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.057820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.057820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.058053 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.058430 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.058431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.058431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.058490 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.058722 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.058723 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.058764 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.059228 (+ 464us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.059314 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":18,"thread_start_us":70,"threads_started":1}
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.126@KRBTEST.COM for oryx/127.19.113.125@KRBTEST.COM
I20260619 02:45:37.067122 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.060183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.19.113.125:44391)
0619 02:45:37.060219 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.060220 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.060336 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:45:37.060439 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.060439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.060446 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.060486 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.060933 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.061384 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.062455 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.062636 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.062690 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.064753 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.064754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.064755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.064761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.066020 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.066386 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.066387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.066388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.066439 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.066602 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.066603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.066603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.066704 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.066757 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.066950 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.067027 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:37.067134 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.060125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.0.0.1:32808)
0619 02:45:37.060243 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.060289 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.060358 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.060493 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.060495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.060813 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.060869 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.060871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.061453 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.061453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.062375 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.062378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.062705 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.062706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.062786 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.063211 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.063216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.064674 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.066108 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.066116 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.066119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.066319 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.066493 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.066494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.066494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.066554 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.066757 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.066766 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.066819 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.066918 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.067040 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":63,"threads_started":1}
I20260619 02:45:37.067703 26006 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.113.125:44391: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2d98ef611cf8467d98094879c3892ab4"
member_type: NON_VOTER
last_known_addr {
host: "127.19.113.125"
port: 44391
}
attrs {
promote: true
}
}
dest_uuid: "145a8ff47a8741ab8202851bc659808f"
cas_config_opid_index: -1
I20260619 02:45:37.068024 26006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } }
I20260619 02:45:37.068495 26566 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2d98ef611cf8467d98094879c3892ab4 (127.19.113.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } }
I20260619 02:45:37.068847 26512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:37.069510 26566 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.113.125:44391
I20260619 02:45:37.069726 26470 master.cc:537] Master@127.19.113.125:44391 shutting down...
I20260619 02:45:37.069816 26512 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Refusing update from remote peer 145a8ff47a8741ab8202851bc659808f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 02:45:37.069726 26565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } }
I20260619 02:45:37.070247 26565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:37.070101 26566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:45:37.071152 25993 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 02:45:37.071627 25993 consensus_peers.cc:487] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f -> Peer 2d98ef611cf8467d98094879c3892ab4 (127.19.113.125:44391): Couldn't send request to peer 2d98ef611cf8467d98094879c3892ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:45:37.072405 26470 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:37.072504 26470 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:37.072552 26470 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: stopping tablet replica
I20260619 02:45:37.085052 26470 master.cc:552] Master@127.19.113.125:44391 shutdown complete.
I20260619 02:45:37.085104 26470 master_runner.cc:305] Clearing existing system tablet
I20260619 02:45:37.085222 26470 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.085263 26470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.085834 26470 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:37.086277 26570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.086400 26470 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.086483 26470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.086509 26470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
uuid: "2d98ef611cf8467d98094879c3892ab4"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "563acf88defdfd431b625316dabc49f1"
server_key_iv: "33d19b62e15099f3d99a927e2bc421a3"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.087347 26470 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:37.088773 26470 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:45:37.088842 26470 log.cc:1196] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 02:45:37.089020 26470 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Deleting consensus metadata
I20260619 02:45:37.089131 26470 master_runner.cc:315] Copying system tablet from 127.19.113.126:32985
I20260619 02:45:37.089721 26470 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.113.126:32985
I20260619 02:45:37.095885 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.090100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.19.113.126:32985)
0619 02:45:37.090150 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.090151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.090180 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:45:37.090257 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.090258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.090266 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.090311 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.090666 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.091222 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.092246 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.092452 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.092505 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.093589 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.093591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.093591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.093597 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.094893 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.095235 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.095236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.095236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.095283 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.095463 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.095463 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.095464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.095579 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.095627 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.095764 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.095822 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:37.095952 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.090007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:40790)
0619 02:45:37.090114 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.090124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.090196 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.090327 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.090328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.090532 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.090601 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.090603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.091290 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.091291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.092157 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.092161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.092519 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.092520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.092605 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.093111 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.093116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.093501 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.094962 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.094963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.094964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.095172 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.095332 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.095333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.095333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.095403 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.095632 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.095633 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.095677 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.095738 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.095838 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":66,"threads_started":1}
I20260619 02:45:37.096328 26026 tablet_copy_service.cc:140] P 145a8ff47a8741ab8202851bc659808f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2d98ef611cf8467d98094879c3892ab4 ({username='oryx', principal='oryx/127.19.113.125@KRBTEST.COM'} at 127.0.0.1:40790)
I20260619 02:45:37.096443 26026 tablet_copy_service.cc:161] P 145a8ff47a8741ab8202851bc659808f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2d98ef611cf8467d98094879c3892ab4 at {username='oryx', principal='oryx/127.19.113.125@KRBTEST.COM'} at 127.0.0.1:40790: session id = 2d98ef611cf8467d98094879c3892ab4-00000000000000000000000000000000
I20260619 02:45:37.097321 26026 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:45:37.097980 26470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:45:37.099088 26470 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: tablet copy: Starting download of 0 data blocks...
I20260619 02:45:37.099225 26470 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: tablet copy: Starting download of 1 WAL segments...
I20260619 02:45:37.100122 26470 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:45:37.100872 26026 tablet_copy_service.cc:342] P 145a8ff47a8741ab8202851bc659808f: Request end of tablet copy session 2d98ef611cf8467d98094879c3892ab4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.113.125@KRBTEST.COM'} at 127.0.0.1:40790
I20260619 02:45:37.100963 26026 tablet_copy_service.cc:433] P 145a8ff47a8741ab8202851bc659808f: ending tablet copy session 2d98ef611cf8467d98094879c3892ab4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2d98ef611cf8467d98094879c3892ab4
I20260619 02:45:37.103085 26470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:37.104166 26577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.104244 26578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.104354 26580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:37.104404 26470 server_base.cc:1034] running on GCE node
I20260619 02:45:37.104535 26470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:37.104658 26470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:37.105772 26470 hybrid_clock.cc:648] HybridClock initialized: now 1781837137105762 us; error 23 us; skew 500 ppm
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:37.107023 26470 init.cc:377] Logged in from keytab as oryx/127.19.113.125@KRBTEST.COM (short username oryx)
I20260619 02:45:37.107266 26470 webserver.cc:460] Webserver started at http://127.19.113.125:45107/ using document root <none> and password file <none>
I20260619 02:45:37.107371 26470 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.107406 26470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.108006 26470 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:37.108486 26587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.108600 26470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.108659 26470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
uuid: "2d98ef611cf8467d98094879c3892ab4"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "563acf88defdfd431b625316dabc49f1"
server_key_iv: "33d19b62e15099f3d99a927e2bc421a3"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.108717 26470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.154425 26470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:37.156132 26470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:37.158917 26470 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.125:44391
I20260619 02:45:37.159190 26639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.125:44391 every 8 connection(s)
I20260619 02:45:37.159278 26470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
I20260619 02:45:37.160037 26640 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:45:37.160485 26640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Bootstrap starting.
I20260619 02:45:37.163183 26640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:45:37.163357 26640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: Bootstrap complete.
I20260619 02:45:37.163544 26640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } }
I20260619 02:45:37.163614 26640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2d98ef611cf8467d98094879c3892ab4, State: Initialized, Role: LEARNER
I20260619 02:45:37.163650 26640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } }
I20260619 02:45:37.163807 26640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:37.163960 26643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } }
I20260619 02:45:37.164045 26643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:45:37.165189 26654 catalog_manager.cc:1259] Loaded cluster ID: 8a5f132352ea4bffa4599117c6d37317
I20260619 02:45:37.165235 26654 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: loading cluster ID for follower catalog manager: success
I20260619 02:45:37.166291 26654 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: acquiring CA information for follower catalog manager: success
I20260619 02:45:37.166565 26654 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:45:37.195163 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.188270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.19.113.125:44391)
0619 02:45:37.188433 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.188434 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.188441 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:37.188463 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.188463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.188472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.188523 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.189278 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.189997 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.191079 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.191281 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.191347 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.192714 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.192715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.192716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.192723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.194105 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.194466 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.194467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.194467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.194521 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.194689 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.194689 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.194689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.194787 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.194852 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.195021 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.195090 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for oryx/127.19.113.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.124.keytab.
Entry for principal oryx/127.19.113.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.124.keytab.
I20260619 02:45:37.219668 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.124:44455
--webserver_interface=127.19.113.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.124.keytab
--principal=oryx/127.19.113.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:32985,127.19.113.125:44391,127.19.113.124:44455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:37.300267 26658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:37.300493 26658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:37.300525 26658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:37.301692 26658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:37.301730 26658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:37.301749 26658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:37.301764 26658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:37.301781 26658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:37.303304 26658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:32985,127.19.113.125:44391,127.19.113.124:44455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.124.keytab
--principal=oryx/127.19.113.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.124:44455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.113.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:37.303548 26658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:37.303755 26658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:45:37.308329 26658 server_base.cc:1034] running on GCE node
W20260619 02:45:37.308303 26664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.308295 26666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.308303 26663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:37.308729 26658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:37.309029 26658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:37.310163 26658 hybrid_clock.cc:648] HybridClock initialized: now 1781837137310157 us; error 36 us; skew 500 ppm
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:37.312568 26658 init.cc:377] Logged in from keytab as oryx/127.19.113.124@KRBTEST.COM (short username oryx)
I20260619 02:45:37.313035 26658 webserver.cc:460] Webserver started at http://127.19.113.124:46271/ using document root <none> and password file <none>
I20260619 02:45:37.313246 26658 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.313288 26658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.313406 26658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:37.314411 26658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data/instance:
uuid: "5dfbbbbafe674ac4a6a206ab6a05550b"
format_stamp: "Formatted at 2026-06-19 02:45:37 on dist-test-slave-zqpb"
server_key: "877986d6f03cd211baad70eeff8cb712"
server_key_iv: "1341f708640b1513b3496ad9122bf2e2"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.314733 26658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal/instance:
uuid: "5dfbbbbafe674ac4a6a206ab6a05550b"
format_stamp: "Formatted at 2026-06-19 02:45:37 on dist-test-slave-zqpb"
server_key: "877986d6f03cd211baad70eeff8cb712"
server_key_iv: "1341f708640b1513b3496ad9122bf2e2"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.316044 26658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:37.316749 26673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.317019 26658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:37.317157 26658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
uuid: "5dfbbbbafe674ac4a6a206ab6a05550b"
format_stamp: "Formatted at 2026-06-19 02:45:37 on dist-test-slave-zqpb"
server_key: "877986d6f03cd211baad70eeff8cb712"
server_key_iv: "1341f708640b1513b3496ad9122bf2e2"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.317245 26658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.324738 26658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:37.326701 26658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:37.326829 26658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:37.330776 26658 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.124:44455
I20260619 02:45:37.330806 26725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.124:44455 every 8 connection(s)
I20260619 02:45:37.331176 26658 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
I20260619 02:45:37.332124 26726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:37.333696 26726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:37.334497 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 26658
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.124@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.113.124@KRBTEST.COM
I20260619 02:45:37.343531 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.334582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.19.113.126:32985)
0619 02:45:37.334633 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.334635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.334759 (+ 124us) server_negotiation.cc:407] Connection header received
0619 02:45:37.335436 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.335438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.335451 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.335501 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.336015 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.336528 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.337543 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.337832 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.337888 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.340296 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.340298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.340298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.340306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.341841 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.342320 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.342321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.342322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.342384 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.342932 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.342933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.342934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.343093 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.343154 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.343357 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.343437 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:45:37.343783 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.334430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:40800)
0619 02:45:37.334636 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.334683 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.335324 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.335528 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.335530 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.335778 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.335928 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.335932 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.336618 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.336619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.337469 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.337471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.337919 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.337920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.338054 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.338662 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.338670 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.340191 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.341954 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.341961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.341963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.342255 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.342716 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.342717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.342717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.342816 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.343152 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.343159 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.343242 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.343323 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.343499 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260619 02:45:37.343784 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.335027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.19.113.124:44455)
0619 02:45:37.335175 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.335179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.335217 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:45:37.335326 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.335336 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.335359 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.335446 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.336047 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.337026 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.337850 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.338256 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.338317 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.340764 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.340772 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.340772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.340779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.342395 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.342701 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.342703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.342704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.342783 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.343019 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.343020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.343021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.343268 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.343419 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.343484 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.343557 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260619 02:45:37.344400 26726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.124@KRBTEST.COM for oryx/127.19.113.125@KRBTEST.COM
I20260619 02:45:37.350344 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.344628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.0.0.1:32838)
0619 02:45:37.344667 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.344677 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.344711 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.344844 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.344845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.344958 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.344995 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.344996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.345619 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.345619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.346229 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.346231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.346590 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.346590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.346635 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.347138 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.347142 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.348202 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.349614 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.349614 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.349615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.349750 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.349894 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.349895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.349895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.349935 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.350133 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.350133 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.350168 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.350211 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.350279 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:45:37.350354 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.344696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.19.113.125:44391)
0619 02:45:37.344738 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.344740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.344747 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:37.344770 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.344771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.344783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.344840 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.345091 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.345565 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.346327 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.346533 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.346580 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.348289 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.348290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.348290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.348297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.349552 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.349812 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.349812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.349813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.349854 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.349998 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.349998 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.349998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.350092 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.350136 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.350228 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.350293 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:37.350790 26726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.124@KRBTEST.COM for oryx/127.19.113.124@KRBTEST.COM
I20260619 02:45:37.356339 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.351105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.19.113.124:44455)
0619 02:45:37.351148 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.351150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.351165 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:45:37.351181 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.351182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.351191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.351228 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.351426 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.351960 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.352560 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.352779 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.352828 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.354281 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.354282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.354283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.354288 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.355407 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.355666 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.355667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.355667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.355709 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.355853 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.355854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.355854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.355951 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.356000 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.356160 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.356246 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:45:37.356413 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.351050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.0.0.1:52136)
0619 02:45:37.351089 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.351099 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.351128 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.351229 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.351230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.351341 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.351366 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.351367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.352036 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.352037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.352500 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.352501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.352831 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.352831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.352874 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.353184 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.353189 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.354215 (+ 1026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.355473 (+ 1258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.355474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.355474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.355613 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.355756 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.355757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.355757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.355806 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.356046 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.356047 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.356081 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.356163 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.356311 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":70}
I20260619 02:45:37.357709 26726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Bootstrap starting.
I20260619 02:45:37.358511 26726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:37.358755 26726 log.cc:824] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:37.359469 26726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: No bootstrap required, opened a new log
I20260619 02:45:37.360863 26726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } }
I20260619 02:45:37.360988 26726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:37.361048 26726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dfbbbbafe674ac4a6a206ab6a05550b, State: Initialized, Role: FOLLOWER
I20260619 02:45:37.361167 26726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } }
I20260619 02:45:37.361387 26731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } } }
I20260619 02:45:37.361459 26726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:37.361474 26731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:45:37.362865 26742 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:37.363094 26742 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:45:37.365321 26658 master_runner.cc:428] Detected that this master 5dfbbbbafe674ac4a6a206ab6a05550b is joining an existing cluster
I20260619 02:45:37.365361 26658 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.113.124:44455
I20260619 02:45:37.372468 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.366543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.0.0.1:52144)
0619 02:45:37.366652 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.366660 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.366695 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.366804 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.366805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.367113 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.367164 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.367165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.368027 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.368028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.368841 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.368843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.369253 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.369254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.369321 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.369671 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.369675 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.370048 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.371609 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.371610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.371610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.371756 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.371906 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.371907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.371907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.371972 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.372163 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.372164 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.372199 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.372257 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.372356 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260619 02:45:37.372520 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.366668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.19.113.124:44455)
0619 02:45:37.366702 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.366703 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.366711 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:37.366732 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.366733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.366741 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.366795 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.367233 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.367940 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.368928 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.369190 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.369246 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.370125 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.370127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.370127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.370133 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.371557 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.371807 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.371808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.371808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.371855 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.372018 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.372019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.372019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.372115 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.372167 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.372284 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.372428 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":17}
I20260619 02:45:37.376724 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.366985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32840 (local address 127.19.113.125:44391)
0619 02:45:37.367026 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.367027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.371011 (+ 3984us) server_negotiation.cc:407] Connection header received
0619 02:45:37.371065 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.371066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.371078 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.371133 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.371467 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.372207 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.373364 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.373556 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.373606 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.374501 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.374502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.374503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.374509 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.375730 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.376072 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.376072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.376073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.376118 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.376297 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.376298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.376298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.376415 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.376461 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.376600 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.376659 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:37.376794 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.366956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.0.0.1:32840)
0619 02:45:37.370948 (+ 3992us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.370958 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.371014 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.371135 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.371136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.371323 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.371391 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.371393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.372299 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.372300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.373293 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.373294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.373620 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.373621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.373685 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.374049 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.374053 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.374419 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.375793 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.375794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.375795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.376009 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.376187 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.376188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.376188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.376247 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.376468 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.376469 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.376513 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.376584 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.376681 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3975,"thread_start_us":77,"threads_started":1}
I20260619 02:45:37.393821 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.366355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.19.113.126:32985)
0619 02:45:37.366406 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.366408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.366530 (+ 122us) server_negotiation.cc:407] Connection header received
0619 02:45:37.366587 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.366588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.366597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.366637 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.367102 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.367648 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.368412 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.368617 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.368669 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.391360 (+ 22691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.391362 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.391362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.391370 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.392751 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.393134 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.393135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.393135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.393178 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.393346 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.393347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.393347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.393461 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.393531 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.393692 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.393757 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:45:37.393913 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.366317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:40810)
0619 02:45:37.366465 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.366476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.366530 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.366686 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.366687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.366934 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.367011 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.367014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.367715 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.367716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.368343 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.368344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.368683 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.368684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.368740 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.369159 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.369163 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.391242 (+ 22079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.392828 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.392830 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.392830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.393069 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.393228 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.393229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.393229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.393293 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.393544 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.393545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.393607 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.393673 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.393786 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260619 02:45:37.395392 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 26068
I20260619 02:45:37.400731 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:43431
--local_ip_for_outbound_sockets=127.19.113.65
--tserver_master_addrs=127.19.113.126:32985
--webserver_port=40371
--webserver_interface=127.19.113.65
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab
--principal=oryx/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391,127.19.113.124:44455 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837136, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.126@KRBTEST.COM for oryx/127.19.113.124@KRBTEST.COM
I20260619 02:45:37.407645 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.400598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.19.113.124:44455)
0619 02:45:37.400635 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.400637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.400646 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:37.400727 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.400728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.400738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.400789 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.401103 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.401736 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.402562 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.402753 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.402809 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.404910 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.404912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.404912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.404920 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.406430 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.406817 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.406818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.406819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.406899 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.407107 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.407107 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.407108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.407241 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.407307 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.407469 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.407549 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:37.407647 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.400542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.0.0.1:52154)
0619 02:45:37.400603 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.400613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.400650 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.400799 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.400800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.400983 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.401034 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.401036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.401839 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.401840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.402486 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.402488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.402823 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.402824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.402906 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.403324 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.403329 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.404806 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.406514 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.406515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.406516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.406733 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.406968 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.406969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.406970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.407041 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.407308 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.407309 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.407358 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.407439 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.407546 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:45:37.408536 26006 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.113.124:44455: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b"
member_type: NON_VOTER
last_known_addr {
host: "127.19.113.124"
port: 44455
}
attrs {
promote: true
}
}
dest_uuid: "145a8ff47a8741ab8202851bc659808f"
cas_config_opid_index: 5
I20260619 02:45:37.408790 26006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } }
I20260619 02:45:37.409179 26565 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.409561 26700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:37.410202 26565 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.113.124:44455
I20260619 02:45:37.410272 26565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.410331 26565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:37.410611 26658 master.cc:537] Master@127.19.113.124:44455 shutting down...
I20260619 02:45:37.410902 26700 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Refusing update from remote peer 145a8ff47a8741ab8202851bc659808f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 02:45:37.411216 26566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:45:37.411813 25991 consensus_peers.cc:487] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f -> Peer 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124:44455): Couldn't send request to peer 5dfbbbbafe674ac4a6a206ab6a05550b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:45:37.413673 26658 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:37.413760 26658 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:37.413800 26658 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: stopping tablet replica
I20260619 02:45:37.415153 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.409205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.0.0.1:32852)
0619 02:45:37.409392 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.409403 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.409446 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.409642 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.409643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.409644 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:37.409677 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.409678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.410541 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.410542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.414009 (+ 3467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.414014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.414890 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.414891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.414967 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.414992 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.415041 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.415081 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260619 02:45:37.415565 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.409264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.19.113.125:44391)
0619 02:45:37.409309 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.409312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.409493 (+ 181us) server_negotiation.cc:407] Connection header received
0619 02:45:37.409519 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.409520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.409541 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.409589 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:37.409733 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.410450 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.414107 (+ 3657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.414690 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.414760 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.414834 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.415448 (+ 614us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.415499 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:45:37.416745 26658 master.cc:552] Master@127.19.113.124:44455 shutdown complete.
I20260619 02:45:37.416791 26658 master_runner.cc:305] Clearing existing system tablet
I20260619 02:45:37.416923 26658 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.416970 26658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.417682 26658 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:37.417726 26614 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 LEARNER]: Refusing update from remote peer 145a8ff47a8741ab8202851bc659808f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 02:45:37.418020 26566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 02:45:37.418637 26614 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.419734 26643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 145a8ff47a8741ab8202851bc659808f. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.419821 26643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:45:37.419852 26643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: NON_VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: true } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.419894 26643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:45:37.420157 26566 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: attempting to promote NON_VOTER 2d98ef611cf8467d98094879c3892ab4 to VOTER
I20260619 02:45:37.420354 26566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } }
I20260619 02:45:37.420925 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.420790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.0.0.1:52164)
0619 02:45:37.420833 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.420886 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 02:45:37.421252 26614 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 LEARNER]: Refusing update from remote peer 145a8ff47a8741ab8202851bc659808f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 02:45:37.421267 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.421172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.0.0.1:52176)
0619 02:45:37.421218 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.421247 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:45:37.421383 25991 consensus_peers.cc:487] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f -> Peer 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124:44455): Couldn't send request to peer 5dfbbbbafe674ac4a6a206ab6a05550b. Status: Network error: Client connection negotiation failed: client connection to 127.19.113.124:44455: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:45:37.421553 26565 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 02:45:37.422116 26565 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2d98ef611cf8467d98094879c3892ab4 (127.19.113.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.422973 26565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.423029 26566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.423116 26566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:37.423045 26565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:37.421939 26754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.423331 26614 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2d98ef611cf8467d98094879c3892ab4 (127.19.113.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.423463 26658 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.423508 26658 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260619 02:45:37.423530 26658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
uuid: "5dfbbbbafe674ac4a6a206ab6a05550b"
format_stamp: "Formatted at 2026-06-19 02:45:37 on dist-test-slave-zqpb"
server_key: "877986d6f03cd211baad70eeff8cb712"
server_key_iv: "1341f708640b1513b3496ad9122bf2e2"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.424268 26658 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:37.424611 26755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.424748 26755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:37.425526 26658 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:45:37.425595 26658 log.cc:1196] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 02:45:37.425772 26658 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Deleting consensus metadata
I20260619 02:45:37.425881 26658 master_runner.cc:315] Copying system tablet from 127.19.113.126:32985
I20260619 02:45:37.426509 26658 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: tablet copy: Beginning tablet copy session from remote peer at address 127.19.113.126:32985
I20260619 02:45:37.433270 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.426985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.19.113.126:32985)
0619 02:45:37.427029 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.427031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.427039 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:37.427060 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.427061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.427071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.427123 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.427458 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.428195 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.429160 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.429457 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.429520 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.430542 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.430543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.430544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.430551 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.432112 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.432503 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.432504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.432505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.432564 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.432755 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.432756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.432756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.432880 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.432940 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.433104 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.433180 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:45:37.433286 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.426766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:40822)
0619 02:45:37.426898 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.426916 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.426973 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.427139 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.427140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.427334 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.427386 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.427388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.428262 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.428263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.429076 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.429080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.429528 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.429530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.429597 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.430077 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.430082 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.430455 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.432198 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.432199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.432200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.432425 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.432624 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.432625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.432626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.432694 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.432944 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.432945 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.432997 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.433073 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.433188 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260619 02:45:37.433812 26026 tablet_copy_service.cc:140] P 145a8ff47a8741ab8202851bc659808f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5dfbbbbafe674ac4a6a206ab6a05550b ({username='oryx', principal='oryx/127.19.113.124@KRBTEST.COM'} at 127.0.0.1:40822)
I20260619 02:45:37.433861 26026 tablet_copy_service.cc:161] P 145a8ff47a8741ab8202851bc659808f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5dfbbbbafe674ac4a6a206ab6a05550b at {username='oryx', principal='oryx/127.19.113.124@KRBTEST.COM'} at 127.0.0.1:40822: session id = 5dfbbbbafe674ac4a6a206ab6a05550b-00000000000000000000000000000000
I20260619 02:45:37.434413 26026 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:45:37.434914 26658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:45:37.436259 26658 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: tablet copy: Starting download of 0 data blocks...
I20260619 02:45:37.436407 26658 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: tablet copy: Starting download of 1 WAL segments...
I20260619 02:45:37.437106 26658 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:45:37.437973 26026 tablet_copy_service.cc:342] P 145a8ff47a8741ab8202851bc659808f: Request end of tablet copy session 5dfbbbbafe674ac4a6a206ab6a05550b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.113.124@KRBTEST.COM'} at 127.0.0.1:40822
I20260619 02:45:37.438030 26026 tablet_copy_service.cc:433] P 145a8ff47a8741ab8202851bc659808f: ending tablet copy session 5dfbbbbafe674ac4a6a206ab6a05550b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5dfbbbbafe674ac4a6a206ab6a05550b
I20260619 02:45:37.440871 26658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:37.442108 26766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.442119 26763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:37.442350 26658 server_base.cc:1034] running on GCE node
W20260619 02:45:37.442363 26764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:37.442562 26658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:37.442699 26658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:37.443812 26658 hybrid_clock.cc:648] HybridClock initialized: now 1781837137443799 us; error 29 us; skew 500 ppm
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:37.445261 26658 init.cc:377] Logged in from keytab as oryx/127.19.113.124@KRBTEST.COM (short username oryx)
I20260619 02:45:37.445530 26658 webserver.cc:460] Webserver started at http://127.19.113.124:45305/ using document root <none> and password file <none>
I20260619 02:45:37.445644 26658 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.445681 26658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.446393 26658 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:37.447006 26773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.447167 26658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:37.447222 26658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
uuid: "5dfbbbbafe674ac4a6a206ab6a05550b"
format_stamp: "Formatted at 2026-06-19 02:45:37 on dist-test-slave-zqpb"
server_key: "877986d6f03cd211baad70eeff8cb712"
server_key_iv: "1341f708640b1513b3496ad9122bf2e2"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.447283 26658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.462203 26658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:37.464020 26658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:37.467190 26658 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.124:44455
I20260619 02:45:37.467582 26658 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
I20260619 02:45:37.468532 26826 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:45:37.469062 26826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Bootstrap starting.
I20260619 02:45:37.470404 26825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.124:44455 every 8 connection(s)
I20260619 02:45:37.472219 26826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:45:37.472446 26826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Bootstrap complete.
I20260619 02:45:37.472641 26826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } }
I20260619 02:45:37.472734 26826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5dfbbbbafe674ac4a6a206ab6a05550b, State: Initialized, Role: LEARNER
I20260619 02:45:37.472774 26826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } }
I20260619 02:45:37.472982 26826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:37.474227 26829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.474316 26829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: This master's current role is: LEARNER
I20260619 02:45:37.474665 26840 catalog_manager.cc:1259] Loaded cluster ID: 8a5f132352ea4bffa4599117c6d37317
I20260619 02:45:37.474740 26840 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: loading cluster ID for follower catalog manager: success
I20260619 02:45:37.475935 26840 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: acquiring CA information for follower catalog manager: success
I20260619 02:45:37.476230 26840 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:45:37.496013 26751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:37.496206 26751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:37.496230 26751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:37.497418 26751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:37.497455 26751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:37.497479 26751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:37.499050 26751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab
--principal=oryx/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:43431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=40371
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391,127.19.113.124:44455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:37.499262 26751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:37.499480 26751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:37.504106 26845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.504108 26846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.504390 26848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:37.504516 26751 server_base.cc:1034] running on GCE node
I20260619 02:45:37.504678 26751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:37.504902 26751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:37.506049 26751 hybrid_clock.cc:648] HybridClock initialized: now 1781837137506037 us; error 31 us; skew 500 ppm
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:37.508388 26751 init.cc:377] Logged in from keytab as oryx/127.19.113.65@KRBTEST.COM (short username oryx)
I20260619 02:45:37.508793 26751 webserver.cc:460] Webserver started at http://127.19.113.65:40371/ using document root <none> and password file <none>
I20260619 02:45:37.508981 26751 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.509033 26751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.510349 26751 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:37.510943 26855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.511091 26751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.511173 26751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "579681295fbc400ea66bae3b061cbfa7"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "37a5acfd1c66f5149a8dbaaa8874eb95"
server_key_iv: "7da40fd37c084aab262dc2d15572301f"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.511574 26751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.532743 26751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:37.534639 26751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:37.534754 26751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:37.534997 26751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:37.535264 26751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:37.535291 26751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.535318 26751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:37.535333 26751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.540642 26751 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:43431
I20260619 02:45:37.540711 26968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:43431 every 8 connection(s)
I20260619 02:45:37.541013 26751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:37.546530 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 26751
I20260619 02:45:37.546684 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 26202
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.124@KRBTEST.COM
I20260619 02:45:37.552095 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.541978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:59859 (local address 127.19.113.126:32985)
0619 02:45:37.542106 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.542108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.542119 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:37.543129 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.543130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.543142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.543172 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.543972 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.544715 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.545723 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.546008 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.546062 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.549104 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.549105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.549106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.549113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.550542 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.550941 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.550942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.550943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.551001 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.551258 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.551259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.551259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.551383 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.551451 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.551739 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.551873 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260619 02:45:37.552165 26974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.541690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:59859)
0619 02:45:37.541912 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.541953 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.543010 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.543259 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.543264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.543692 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.543859 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.543865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.544796 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.544797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.545653 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.545655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.546115 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.546116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.546180 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.546780 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.546788 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.549008 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.550615 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.550621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.550624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.550834 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.551056 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.551057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.551057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.551174 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.551447 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.551454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.551604 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.551690 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.551937 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":121,"thread_start_us":143,"threads_started":1}
I20260619 02:45:37.552362 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:42895
--local_ip_for_outbound_sockets=127.19.113.66
--tserver_master_addrs=127.19.113.126:32985
--webserver_port=38955
--webserver_interface=127.19.113.66
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab
--principal=oryx/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391,127.19.113.124:44455 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:45:37.552561 26970 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32985
I20260619 02:45:37.552593 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.541690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.19.113.65:52841)
0619 02:45:37.541931 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.541954 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.543012 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.543260 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.543264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.543655 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.543859 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.543863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.545016 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.545017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.545614 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.545616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.546069 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.546070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.546177 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.547445 (+ 1268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.547450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.549011 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.550860 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.550861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.550861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.551047 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.551210 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.551211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.551211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.551264 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.551512 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.551513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.551606 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.551691 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.551865 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260619 02:45:37.552762 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.541988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:52841 (local address 127.19.113.124:44455)
0619 02:45:37.542163 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.542165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.542172 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:37.543104 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.543105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.543120 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.543185 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.543945 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.544949 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.545688 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.545999 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.546062 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.549360 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.549361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.549361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.549369 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.550786 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.551102 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.551103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.551103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.551156 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.551331 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.551332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.551333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.551456 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.551520 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.552568 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.552661 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260619 02:45:37.553110 26970 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.553128 26969 heartbeater.cc:347] Connected to a master server at 127.19.113.124:44455
I20260619 02:45:37.553224 26969 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.553399 26969 heartbeater.cc:511] Master 127.19.113.124:44455 requested a full tablet report, sending...
I20260619 02:45:37.553592 26970 heartbeater.cc:511] Master 127.19.113.126:32985 requested a full tablet report, sending...
I20260619 02:45:37.553835 26006 ts_manager.cc:194] Re-registered known tserver with Master: 579681295fbc400ea66bae3b061cbfa7 (127.19.113.65:43431)
I20260619 02:45:37.553905 26790 ts_manager.cc:194] Registered new tserver with Master: 579681295fbc400ea66bae3b061cbfa7 (127.19.113.65:43431)
I20260619 02:45:37.554302 26006 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:59859
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.125@KRBTEST.COM
I20260619 02:45:37.557988 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.541690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.19.113.65:46259)
0619 02:45:37.542211 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.542219 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.543016 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.543259 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.543264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.543698 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.543859 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.543865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.544541 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.544541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.545663 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.545664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.546038 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.546039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.546177 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.553651 (+ 7474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.553655 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.555319 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.557098 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.557099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.557099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.557258 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.557457 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.557457 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.557457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.557503 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.557754 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.557754 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.557786 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.557831 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.557914 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":47,"spinlock_wait_cycles":6912,"thread_start_us":94,"threads_started":1}
I20260619 02:45:37.558022 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.541730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:46259 (local address 127.19.113.125:44391)
0619 02:45:37.541787 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.541789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.542288 (+ 499us) server_negotiation.cc:407] Connection header received
0619 02:45:37.543129 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.543130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.543142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.543172 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.543969 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.544448 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.545748 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.545940 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.545969 (+ 29us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.555424 (+ 9455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.555426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.555426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.555435 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.557006 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.557330 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.557331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.557331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.557392 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.557554 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.557555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.557555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.557698 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.557768 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.557856 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.557932 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:45:37.558401 26971 heartbeater.cc:347] Connected to a master server at 127.19.113.125:44391
I20260619 02:45:37.558451 26971 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.558524 26971 heartbeater.cc:511] Master 127.19.113.125:44391 requested a full tablet report, sending...
I20260619 02:45:37.558931 26604 ts_manager.cc:194] Registered new tserver with Master: 579681295fbc400ea66bae3b061cbfa7 (127.19.113.65:43431)
W20260619 02:45:37.632808 26977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:37.633028 26977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:37.633055 26977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:37.634191 26977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:37.634231 26977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:37.634256 26977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:37.635802 26977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab
--principal=oryx/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:42895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=38955
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391,127.19.113.124:44455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:37.636026 26977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:37.636220 26977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:37.640882 26985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.640856 26983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.641031 26982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:37.640986 26977 server_base.cc:1034] running on GCE node
I20260619 02:45:37.641281 26977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:37.641503 26977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:37.642652 26977 hybrid_clock.cc:648] HybridClock initialized: now 1781837137642637 us; error 34 us; skew 500 ppm
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:37.645058 26977 init.cc:377] Logged in from keytab as oryx/127.19.113.66@KRBTEST.COM (short username oryx)
I20260619 02:45:37.645452 26977 webserver.cc:460] Webserver started at http://127.19.113.66:38955/ using document root <none> and password file <none>
I20260619 02:45:37.645655 26977 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.645709 26977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.647032 26977 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:37.647605 26992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.647727 26977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.647805 26977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "ec3dcb4e39e0428c8ca53257cf94b645"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "eb6c435a6997b465c8274515237033bf"
server_key_iv: "b8c108792c4a3a9158c9c5600abfb53c"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.648131 26977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.662213 26977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:37.664453 26977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:37.664575 26977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:37.664801 26977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:37.665132 26977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:37.665176 26977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.665215 26977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:37.665235 26977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.670800 26977 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:42895
I20260619 02:45:37.670889 27105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:42895 every 8 connection(s)
I20260619 02:45:37.671222 26977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:37.678004 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 26977
I20260619 02:45:37.678172 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 26336
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.125@KRBTEST.COM
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.124@KRBTEST.COM
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:37.683460 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.672043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:51845 (local address 127.19.113.126:32985)
0619 02:45:37.672198 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.672200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.672294 (+ 94us) server_negotiation.cc:407] Connection header received
0619 02:45:37.673423 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.673424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.673437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.673493 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.674339 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.675141 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.677070 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.677314 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.677386 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.680211 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.680212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.680213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.680221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.681888 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.682343 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.682345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.682345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.682415 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.682613 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.682614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.682614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.682746 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.682786 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.683265 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.683356 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260619 02:45:37.683528 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.672094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:39085 (local address 127.19.113.125:44391)
0619 02:45:37.672141 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.672143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.672476 (+ 333us) server_negotiation.cc:407] Connection header received
0619 02:45:37.673459 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.673461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.673474 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.673502 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.674348 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.675141 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.676661 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.676842 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.676893 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.680211 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.680212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.680212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.680221 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.681737 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.682276 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.682278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.682278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.682340 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.682605 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.682606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.682607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.682744 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.682782 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.683343 (+ 561us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.683441 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:37.683818 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:40713
--local_ip_for_outbound_sockets=127.19.113.67
--tserver_master_addrs=127.19.113.126:32985
--webserver_port=39621
--webserver_interface=127.19.113.67
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab
--principal=oryx/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391,127.19.113.124:44455 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:45:37.684092 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.671932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.19.113.66:39085)
0619 02:45:37.672409 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.672418 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.673341 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.673576 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.673577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.673981 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.674217 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.674223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.675214 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.675215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.676262 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.676265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.676913 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.676914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.676983 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.677373 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.677378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.680096 (+ 2718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.681827 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.681835 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.681838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.682192 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.682404 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.682405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.682405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.682539 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.682822 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.682912 (+ 90us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.683103 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.683801 (+ 698us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.683918 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":41,"thread_start_us":82,"threads_started":1}
I20260619 02:45:37.684092 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.671924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.66:51845)
0619 02:45:37.672176 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.672219 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.673315 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.673556 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.673559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.674004 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.674216 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.674221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.676358 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.676358 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.676999 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.677001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.677438 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.677439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.677505 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.677852 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.677856 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.680096 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.681977 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.681978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.681978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.682252 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.682488 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.682489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.682489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.682546 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.682942 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.682942 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.683102 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.683599 (+ 497us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.683790 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260619 02:45:37.684700 27107 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32985
I20260619 02:45:37.684799 27107 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.685006 27107 heartbeater.cc:511] Master 127.19.113.126:32985 requested a full tablet report, sending...
I20260619 02:45:37.685349 27108 heartbeater.cc:347] Connected to a master server at 127.19.113.125:44391
I20260619 02:45:37.685403 27108 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.685470 27108 heartbeater.cc:511] Master 127.19.113.125:44391 requested a full tablet report, sending...
I20260619 02:45:37.685472 26006 ts_manager.cc:194] Re-registered known tserver with Master: ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895)
I20260619 02:45:37.685760 26604 ts_manager.cc:194] Registered new tserver with Master: ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895)
I20260619 02:45:37.685904 26006 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:51845
I20260619 02:45:37.686393 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.671924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.19.113.66:59659)
0619 02:45:37.672242 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.672250 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.673321 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.673645 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.673646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.673998 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.674217 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.674223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.675032 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.675032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.676262 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.676265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.676632 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.676633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.676763 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.677279 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.677289 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.680349 (+ 3060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.682812 (+ 2463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.682813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.682813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.683028 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.683233 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.683233 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.683234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.683284 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.686079 (+ 2795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.686080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.686127 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.686187 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.686294 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":24,"thread_start_us":73,"threads_started":1}
I20260619 02:45:37.686420 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.671988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:59659 (local address 127.19.113.124:44455)
0619 02:45:37.672158 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.672160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.672309 (+ 149us) server_negotiation.cc:407] Connection header received
0619 02:45:37.673552 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.673553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.673563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.673614 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.674327 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.674915 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.676348 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.676558 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.676611 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.680825 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.680827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.680827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.680836 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.682731 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.683106 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.683107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.683108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.683165 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.683348 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.683349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.683349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.683483 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.683523 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.686233 (+ 2710us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.686325 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260619 02:45:37.686712 27106 heartbeater.cc:347] Connected to a master server at 127.19.113.124:44455
I20260619 02:45:37.686758 27106 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.686950 27106 heartbeater.cc:511] Master 127.19.113.124:44455 requested a full tablet report, sending...
I20260619 02:45:37.687209 26790 ts_manager.cc:194] Registered new tserver with Master: ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895)
W20260619 02:45:37.763716 27113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:37.763971 27113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:37.764034 27113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:37.765252 27113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:37.765291 27113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:37.765316 27113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:37.766770 27113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab
--principal=oryx/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:40713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=39621
--tserver_master_addrs=127.19.113.126:32985,127.19.113.125:44391,127.19.113.124:44455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:37.767020 27113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:37.767233 27113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:37.771942 27118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:37.772118 27113 server_base.cc:1034] running on GCE node
W20260619 02:45:37.771981 27119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:37.771981 27121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:37.772380 27113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:37.772601 27113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:37.773741 27113 hybrid_clock.cc:648] HybridClock initialized: now 1781837137773726 us; error 28 us; skew 500 ppm
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:37.776160 27113 init.cc:377] Logged in from keytab as oryx/127.19.113.67@KRBTEST.COM (short username oryx)
I20260619 02:45:37.776521 27113 webserver.cc:460] Webserver started at http://127.19.113.67:39621/ using document root <none> and password file <none>
I20260619 02:45:37.776705 27113 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:37.776751 27113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:37.777988 27113 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:37.778471 27128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.778614 27113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.778684 27113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6"
format_stamp: "Formatted at 2026-06-19 02:45:36 on dist-test-slave-zqpb"
server_key: "cc9e42730e6a05df898dfc61f8663df6"
server_key_iv: "b3877622cd8430394bb85e5a76b1d9d1"
server_key_version: "encryptionkey@0"
I20260619 02:45:37.779083 27113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:37.802205 27113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:37.804296 27113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:37.804416 27113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:37.804646 27113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:37.804924 27113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:37.804955 27113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.804976 27113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:37.804999 27113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:37.810354 27113 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:40713
I20260619 02:45:37.810421 27241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:40713 every 8 connection(s)
I20260619 02:45:37.810703 27113 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.124@KRBTEST.COM
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
Jun 19 02:45:37 dist-test-slave-zqpb krb5kdc[25961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837137, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.125@KRBTEST.COM
I20260619 02:45:37.819195 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 27113
I20260619 02:45:37.821823 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.811668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:57919 (local address 127.19.113.124:44455)
0619 02:45:37.811785 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.811787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.811794 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:37.812706 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.812707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.812719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.812769 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.813508 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.814273 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.815254 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.815534 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.815600 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.817892 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.817893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.817894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.817902 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.819382 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.819865 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.819866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.819867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.819936 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.821067 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.821068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.821068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.821212 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.821252 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.821642 (+ 390us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.821723 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 02:45:37.821985 27248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.811392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.19.113.67:43705)
0619 02:45:37.811898 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.811906 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.812613 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.812979 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.812980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.813259 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.813419 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.813425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.814096 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.814097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.815247 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.815250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.815709 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.815710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.815769 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.816336 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.816341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.818961 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.820674 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.820675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.820675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.820902 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.821106 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.821107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.821107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.821167 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.821427 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.821428 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.821475 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.821581 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.821745 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":10,"thread_start_us":83,"threads_started":1}
I20260619 02:45:37.822376 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.811652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:43705 (local address 127.19.113.125:44391)
0619 02:45:37.811743 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.811745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.811940 (+ 195us) server_negotiation.cc:407] Connection header received
0619 02:45:37.812901 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.812903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.812915 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.812965 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.813527 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.814005 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.815365 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.815651 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.815709 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.819090 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.819091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.819092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.819099 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.820504 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.820980 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.820982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.820982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.821041 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.821229 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.821229 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.821230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.821351 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.821403 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.822214 (+ 811us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.822287 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:45:37.822670 27244 heartbeater.cc:347] Connected to a master server at 127.19.113.125:44391
I20260619 02:45:37.822767 27244 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.822984 27244 heartbeater.cc:511] Master 127.19.113.125:44391 requested a full tablet report, sending...
I20260619 02:45:37.823323 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.811618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:48293 (local address 127.19.113.126:32985)
0619 02:45:37.811705 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.811708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.811862 (+ 154us) server_negotiation.cc:407] Connection header received
0619 02:45:37.812781 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.812783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.812795 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.812843 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.813533 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.814277 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.815755 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.816020 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.816078 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.818187 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.818188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.818189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.818196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.820170 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.820543 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.820545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.820545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.820607 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.822763 (+ 2156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.822764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.822764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.822929 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.822973 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.823162 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.823237 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:45:37.823701 26604 ts_manager.cc:194] Registered new tserver with Master: dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713)
I20260619 02:45:37.825409 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.811392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.19.113.67:57919)
0619 02:45:37.811554 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.811594 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.812602 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.812837 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.812841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.813221 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.813418 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.813423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.814379 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.814379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.815177 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.815180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.815607 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.815607 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.815754 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.816306 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.816316 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.817795 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.819510 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.819516 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.819518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.819793 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.820862 (+ 1069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.820863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.820863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.820979 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.821295 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.821301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.821472 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.823281 (+ 1809us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.825285 (+ 2004us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260619 02:45:37.825920 27242 heartbeater.cc:347] Connected to a master server at 127.19.113.124:44455
I20260619 02:45:37.825970 27242 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.826037 27242 heartbeater.cc:511] Master 127.19.113.124:44455 requested a full tablet report, sending...
I20260619 02:45:37.826303 26790 ts_manager.cc:194] Registered new tserver with Master: dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713)
I20260619 02:45:37.827572 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.811393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:48293)
0619 02:45:37.811811 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.811820 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.812609 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.812866 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.812867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.813245 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:37.813419 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.813425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.814354 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.814355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.815250 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.815251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.816082 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.816083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.816139 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.816465 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:37.816470 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:37.818116 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:37.820254 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.820255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.820256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.820459 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.820688 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:37.820688 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:37.820689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:37.822688 (+ 1999us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:37.823019 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:37.823020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:37.823058 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.824003 (+ 945us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.827259 (+ 3256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":3234,"thread_start_us":78,"threads_started":1}
I20260619 02:45:37.827970 27243 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32985
I20260619 02:45:37.828014 27243 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:37.828078 27243 heartbeater.cc:511] Master 127.19.113.126:32985 requested a full tablet report, sending...
I20260619 02:45:37.828297 26006 ts_manager.cc:194] Re-registered known tserver with Master: dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713)
I20260619 02:45:37.828709 26006 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:48293
I20260619 02:45:37.829703 27256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.820746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.19.113.126:32985)
0619 02:45:37.820872 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.820874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.821093 (+ 219us) server_negotiation.cc:407] Connection header received
0619 02:45:37.821179 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.821180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.821191 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.821234 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.822093 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.822851 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.823832 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.824148 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.824183 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.825497 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.825498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.825499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.825506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.827225 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.827585 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.827587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.827587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.827672 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.828999 (+ 1327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.829001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.829001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.829118 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.829174 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.829538 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.829615 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":81,"threads_started":1}
I20260619 02:45:37.830072 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.820586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53662 (local address 127.19.113.125:44391)
0619 02:45:37.821067 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.821069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.821088 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:45:37.821105 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.821106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.821117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.822002 (+ 885us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.822039 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.823235 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.824138 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.824321 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.824368 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.825774 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.825775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.825775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.825781 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.828255 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.828663 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.828665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.828665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.828731 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.829356 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.829356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.829357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.829484 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.829856 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.829893 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.829982 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":50,"threads_started":1}
I20260619 02:45:37.830215 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.820801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.19.113.124:44455)
0619 02:45:37.821893 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.821895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.821901 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:45:37.821919 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.821919 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.821928 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.821976 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.823584 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.824248 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.825248 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.825558 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.825637 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.827344 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.827345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.827345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.827352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.828921 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.829411 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.829411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.829412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.829462 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.829672 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.829672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.829673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.829803 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.829867 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.830065 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.830130 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1080,"thread_start_us":54,"threads_started":1}
I20260619 02:45:37.831197 26006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:37.832682 26006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:37.840430 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.836407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40713 (local address 127.0.0.1:48982)
0619 02:45:37.836598 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.836607 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.836650 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.836902 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.836902 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.836904 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:37.836946 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.836947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.838083 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.838084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.839319 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.839322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.840189 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.840190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.840243 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.840247 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.840290 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.840332 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":42,"thread_start_us":54,"threads_started":1}
I20260619 02:45:37.840440 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.836485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48982 (local address 127.19.113.67:40713)
0619 02:45:37.836749 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.836753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.836761 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:37.836793 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.836795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.836809 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.836902 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:37.836997 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.838003 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.839408 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.840111 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.840175 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.840245 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.840322 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.840365 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":101,"threads_started":1}
I20260619 02:45:37.841096 27176 tablet_service.cc:1467] Processing CreateTablet for tablet bfd39ab581d34a10b11b36a51fde5eea (DEFAULT_TABLE table=test-table [id=afab5d8377774d59bfcbe5274a601ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:37.841290 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.836178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:42895 (local address 127.0.0.1:45092)
0619 02:45:37.836212 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.836223 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.836261 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.836717 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.836718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.836720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:37.836763 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.836765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.838143 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.838143 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.839319 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.839322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.841072 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.841073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.841133 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.841136 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.841196 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.841234 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":5}
I20260619 02:45:37.841432 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.836350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.19.113.66:42895)
0619 02:45:37.836549 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.836554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.836563 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:37.836604 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.836606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.836623 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.836856 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:37.836914 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.838068 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.840281 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.840992 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.841063 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.841133 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.841296 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.841355 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":84,"threads_started":1}
I20260619 02:45:37.841461 27176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfd39ab581d34a10b11b36a51fde5eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:37.842038 27040 tablet_service.cc:1467] Processing CreateTablet for tablet bfd39ab581d34a10b11b36a51fde5eea (DEFAULT_TABLE table=test-table [id=afab5d8377774d59bfcbe5274a601ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:37.842190 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.836226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:43431 (local address 127.0.0.1:56154)
0619 02:45:37.836341 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.836349 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.836392 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.838262 (+ 1870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.838263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.838265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:37.838307 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.838309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.839206 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.839207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.840248 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.840251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.841972 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.841973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.842033 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.842036 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.842088 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.842126 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":84,"threads_started":1}
I20260619 02:45:37.842350 27040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfd39ab581d34a10b11b36a51fde5eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:37.842386 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.836631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56154 (local address 127.19.113.65:43431)
0619 02:45:37.838122 (+ 1491us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.838126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.838135 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:37.838169 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.838171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.838185 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.838251 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:37.838363 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.839144 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.840332 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.841895 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.841956 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.842020 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.842102 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.842138 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":73,"threads_started":1}
I20260619 02:45:37.842849 26903 tablet_service.cc:1467] Processing CreateTablet for tablet bfd39ab581d34a10b11b36a51fde5eea (DEFAULT_TABLE table=test-table [id=afab5d8377774d59bfcbe5274a601ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:37.843155 26903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfd39ab581d34a10b11b36a51fde5eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:37.844882 27268 tablet_bootstrap.cc:492] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: Bootstrap starting.
I20260619 02:45:37.845498 27268 tablet_bootstrap.cc:654] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:37.845498 27269 tablet_bootstrap.cc:492] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Bootstrap starting.
I20260619 02:45:37.845839 27268 log.cc:824] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:37.845966 27270 tablet_bootstrap.cc:492] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: Bootstrap starting.
I20260619 02:45:37.846314 27269 tablet_bootstrap.cc:654] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:37.846553 27270 tablet_bootstrap.cc:654] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:37.846664 27268 tablet_bootstrap.cc:492] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: No bootstrap required, opened a new log
I20260619 02:45:37.846628 27269 log.cc:824] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:37.846727 27268 ts_tablet_manager.cc:1397] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:37.846854 27270 log.cc:824] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:37.847404 27269 tablet_bootstrap.cc:492] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: No bootstrap required, opened a new log
I20260619 02:45:37.847468 27269 ts_tablet_manager.cc:1397] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:37.847587 27270 tablet_bootstrap.cc:492] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: No bootstrap required, opened a new log
I20260619 02:45:37.847646 27270 ts_tablet_manager.cc:1397] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:45:37.849133 27269 raft_consensus.cc:348] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.849259 27269 raft_consensus.cc:374] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:37.849289 27269 raft_consensus.cc:729] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3dcb4e39e0428c8ca53257cf94b645, State: Initialized, Role: FOLLOWER
I20260619 02:45:37.849241 27268 raft_consensus.cc:348] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.849399 27268 raft_consensus.cc:374] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:37.849406 27269 consensus_queue.cc:260] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.849444 27268 raft_consensus.cc:729] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfcce9bce66242dd9e4df6ebf3a29ff6, State: Initialized, Role: FOLLOWER
I20260619 02:45:37.849567 27268 consensus_queue.cc:260] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.849647 27269 ts_tablet_manager.cc:1428] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:37.849727 27107 heartbeater.cc:503] Master 127.19.113.126:32985 was elected leader, sending a full tablet report...
I20260619 02:45:37.850390 27270 raft_consensus.cc:348] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.850533 27270 raft_consensus.cc:374] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:37.850575 27270 raft_consensus.cc:729] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579681295fbc400ea66bae3b061cbfa7, State: Initialized, Role: FOLLOWER
I20260619 02:45:37.850690 27268 ts_tablet_manager.cc:1428] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:45:37.850694 27270 consensus_queue.cc:260] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.850950 27270 ts_tablet_manager.cc:1428] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:45:37.850997 27243 heartbeater.cc:503] Master 127.19.113.126:32985 was elected leader, sending a full tablet report...
I20260619 02:45:37.851445 26970 heartbeater.cc:503] Master 127.19.113.126:32985 was elected leader, sending a full tablet report...
I20260619 02:45:37.856523 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.854266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.19.113.124:44455)
0619 02:45:37.854317 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.854319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.854328 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:37.854367 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.854367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.854385 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.854433 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:37.854506 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.855005 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.855843 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.856268 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.856326 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.856370 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.856449 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.856474 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:45:37.856525 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.854208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.0.0.1:40200)
0619 02:45:37.854264 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.854275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.854317 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.854434 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.854434 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.854435 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:37.854473 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.854474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.855067 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.855067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.855775 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.855777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.856358 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.856359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.856403 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.856405 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.856445 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.856476 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:45:37.856968 26792 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 02:45:37.857757 26829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 145a8ff47a8741ab8202851bc659808f. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.857851 26829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: This master's current role is: LEARNER
I20260619 02:45:37.910787 27276 raft_consensus.cc:482] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:37.910924 27276 raft_consensus.cc:504] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.911240 27276 leader_election.cc:288] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895), dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713)
I20260619 02:45:37.913898 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.911502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:57735 (local address 127.19.113.67:40713)
0619 02:45:37.911558 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.911560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.911567 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:37.911594 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.911595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.911603 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.911649 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:37.911781 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.912263 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.913210 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.913589 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.913651 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.913660 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.913792 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.913832 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:45:37.913940 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.911433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40713 (local address 127.19.113.65:57735)
0619 02:45:37.911493 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.911502 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.911541 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.911657 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.911658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.911659 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:37.911726 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.911729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.912332 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.912333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.913121 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.913124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.913656 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.913657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.913736 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.913739 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.913782 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.913820 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 02:45:37.913932 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.911575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:43621 (local address 127.19.113.66:42895)
0619 02:45:37.911616 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.911618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.911624 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:45:37.911644 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.911645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.911651 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.911692 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:37.911825 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.912276 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.913229 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.913611 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.913670 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.913678 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.913824 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.913864 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:37.913957 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.911381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:42895 (local address 127.19.113.65:43621)
0619 02:45:37.911447 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:37.911458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:37.911506 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:37.911707 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:37.911707 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:37.911709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:37.911766 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.911767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.912341 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.912342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.913121 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:37.913123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.913678 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.913680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:37.913763 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.913766 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:37.913809 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:37.913850 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":38}
I20260619 02:45:37.914263 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfd39ab581d34a10b11b36a51fde5eea" candidate_uuid: "579681295fbc400ea66bae3b061cbfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" is_pre_election: true
I20260619 02:45:37.914259 27060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfd39ab581d34a10b11b36a51fde5eea" candidate_uuid: "579681295fbc400ea66bae3b061cbfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" is_pre_election: true
I20260619 02:45:37.914405 27196 raft_consensus.cc:2454] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579681295fbc400ea66bae3b061cbfa7 in term 0.
I20260619 02:45:37.914407 27060 raft_consensus.cc:2454] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579681295fbc400ea66bae3b061cbfa7 in term 0.
I20260619 02:45:37.914615 26856 leader_election.cc:302] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579681295fbc400ea66bae3b061cbfa7, dfcce9bce66242dd9e4df6ebf3a29ff6; no voters:
I20260619 02:45:37.914791 27276 raft_consensus.cc:2781] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:37.914829 27276 raft_consensus.cc:482] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:37.914849 27276 raft_consensus.cc:3037] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:37.915576 27276 raft_consensus.cc:504] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.915696 27276 leader_election.cc:288] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [CANDIDATE]: Term 1 election: Requested vote from peers ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895), dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713)
I20260619 02:45:37.915817 27060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfd39ab581d34a10b11b36a51fde5eea" candidate_uuid: "579681295fbc400ea66bae3b061cbfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3dcb4e39e0428c8ca53257cf94b645"
I20260619 02:45:37.915860 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfd39ab581d34a10b11b36a51fde5eea" candidate_uuid: "579681295fbc400ea66bae3b061cbfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6"
I20260619 02:45:37.915918 27060 raft_consensus.cc:3037] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:37.915973 27196 raft_consensus.cc:3037] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:37.916757 27196 raft_consensus.cc:2454] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579681295fbc400ea66bae3b061cbfa7 in term 1.
I20260619 02:45:37.916757 27060 raft_consensus.cc:2454] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579681295fbc400ea66bae3b061cbfa7 in term 1.
I20260619 02:45:37.916908 26856 leader_election.cc:302] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579681295fbc400ea66bae3b061cbfa7, dfcce9bce66242dd9e4df6ebf3a29ff6; no voters:
I20260619 02:45:37.917009 27276 raft_consensus.cc:2781] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:37.917161 27276 raft_consensus.cc:686] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 1 LEADER]: Becoming Leader. State: Replica: 579681295fbc400ea66bae3b061cbfa7, State: Running, Role: LEADER
I20260619 02:45:37.917245 27276 consensus_queue.cc:237] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } }
I20260619 02:45:37.918052 26006 catalog_manager.cc:5495] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 579681295fbc400ea66bae3b061cbfa7 (127.19.113.65). New cstate: current_term: 1 leader_uuid: "579681295fbc400ea66bae3b061cbfa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } health_report { overall_health: UNKNOWN } } }
I20260619 02:45:37.918607 26756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: NON_VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: true } } }
I20260619 02:45:37.918712 26756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:37.919245 26565 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: attempting to promote NON_VOTER 5dfbbbbafe674ac4a6a206ab6a05550b to VOTER
I20260619 02:45:37.919382 26565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } }
I20260619 02:45:37.919757 26614 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Refusing update from remote peer 145a8ff47a8741ab8202851bc659808f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 02:45:37.919790 26792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 LEARNER]: Refusing update from remote peer 145a8ff47a8741ab8202851bc659808f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 02:45:37.919951 26756 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260619 02:45:37.920039 26566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260619 02:45:37.920603 26756 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:37.920732 26614 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:37.920967 26792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:37.921542 26756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:37.921617 26756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:37.921573 26755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:37.921654 26755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:37.921666 26756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:37.921708 26756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:37.921766 26756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:37.921808 26756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:37.921906 26829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "145a8ff47a8741ab8202851bc659808f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
W20260619 02:45:37.921969 27109 tablet.cc:2369] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:37.921990 26829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:37.931320 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.928821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.19.113.65:43431)
0619 02:45:37.928873 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.928875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.928882 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:37.928967 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.928968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.928985 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.929036 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:37.929255 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.929735 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.930498 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.930702 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.930757 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.931024 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:37.931126 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:37.931187 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.931220 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.931243 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:45:37.933116 27060 raft_consensus.cc:1261] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 1 FOLLOWER]: Refusing update from remote peer 579681295fbc400ea66bae3b061cbfa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:37.933116 27196 raft_consensus.cc:1261] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 1 FOLLOWER]: Refusing update from remote peer 579681295fbc400ea66bae3b061cbfa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:37.933462 27281 consensus_queue.cc:1035] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:37.933586 27276 consensus_queue.cc:1035] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:37.936863 27284 mvcc.cc:205] Tried to move back new op lower bound from 7298404916971347968 to 7298404916909690880. Current Snapshot: MvccSnapshot[applied={T|T < 7298404916971347968}]
I20260619 02:45:37.937783 27286 mvcc.cc:205] Tried to move back new op lower bound from 7298404916971347968 to 7298404916909690880. Current Snapshot: MvccSnapshot[applied={T|T < 7298404916971347968 or (T in {7298404916971347968})}]
I20260619 02:45:37.937901 27285 mvcc.cc:205] Tried to move back new op lower bound from 7298404916971347968 to 7298404916909690880. Current Snapshot: MvccSnapshot[applied={T|T < 7298404916971347968}]
I20260619 02:45:37.939503 26006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:37.939584 26006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:37.940713 26006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 145a8ff47a8741ab8202851bc659808f: Sending DeleteTablet for 3 replicas of tablet bfd39ab581d34a10b11b36a51fde5eea
I20260619 02:45:37.941077 27176 tablet_service.cc:1514] Processing DeleteTablet for tablet bfd39ab581d34a10b11b36a51fde5eea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:37 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:48982
I20260619 02:45:37.941119 27040 tablet_service.cc:1514] Processing DeleteTablet for tablet bfd39ab581d34a10b11b36a51fde5eea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:37 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:45092
I20260619 02:45:37.941299 26903 tablet_service.cc:1514] Processing DeleteTablet for tablet bfd39ab581d34a10b11b36a51fde5eea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:37 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:56154
I20260619 02:45:37.941514 27296 tablet_replica.cc:330] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: stopping tablet replica
I20260619 02:45:37.941615 27296 raft_consensus.cc:2229] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:37.941715 27296 raft_consensus.cc:2258] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:37.942131 27296 ts_tablet_manager.cc:1905] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:37.942214 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 25974
I20260619 02:45:37.942348 27295 tablet_replica.cc:330] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: stopping tablet replica
I20260619 02:45:37.942435 27295 raft_consensus.cc:2229] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:37.942520 27295 raft_consensus.cc:2258] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:37.942955 27295 ts_tablet_manager.cc:1905] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:37.943758 27296 ts_tablet_manager.cc:1918] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:37.943818 27296 log.cc:1196] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/bfd39ab581d34a10b11b36a51fde5eea
I20260619 02:45:37.944077 27296 ts_tablet_manager.cc:1939] T bfd39ab581d34a10b11b36a51fde5eea P 579681295fbc400ea66bae3b061cbfa7: Deleting consensus metadata
I20260619 02:45:37.944458 27295 ts_tablet_manager.cc:1918] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:37.944516 27295 log.cc:1196] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/bfd39ab581d34a10b11b36a51fde5eea
I20260619 02:45:37.944726 27295 ts_tablet_manager.cc:1939] T bfd39ab581d34a10b11b36a51fde5eea P ec3dcb4e39e0428c8ca53257cf94b645: Deleting consensus metadata
I20260619 02:45:37.944896 27294 tablet_replica.cc:330] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: stopping tablet replica
I20260619 02:45:37.944998 27294 raft_consensus.cc:2229] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:37.945082 27294 raft_consensus.cc:2258] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:37.945703 27294 ts_tablet_manager.cc:1905] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:37.947165 27294 ts_tablet_manager.cc:1918] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:37.947222 27294 log.cc:1196] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/bfd39ab581d34a10b11b36a51fde5eea
I20260619 02:45:37.947491 27294 ts_tablet_manager.cc:1939] T bfd39ab581d34a10b11b36a51fde5eea P dfcce9bce66242dd9e4df6ebf3a29ff6: Deleting consensus metadata
W20260619 02:45:37.948868 27129 connection.cc:535] server connection from 127.0.0.1:48982 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 02:45:37.948864 26856 connection.cc:535] server connection from 127.0.0.1:56154 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 02:45:37.949162 26995 connection.cc:535] server connection from 127.0.0.1:45092 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:45:37.957170 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.941672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.19.113.124:44455)
0619 02:45:37.941712 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.941715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.944223 (+ 2508us) server_negotiation.cc:407] Connection header received
0619 02:45:37.944249 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.944251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.944262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.944316 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.950250 (+ 5934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.950979 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.951988 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.952269 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.952323 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.953786 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.953787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.953788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.953795 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.955758 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.956122 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.956123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.956124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.956180 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.956363 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.956363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.956364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.956489 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.956565 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.956961 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.957075 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:37.958467 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.950653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.19.113.125:44391)
0619 02:45:37.950691 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.950692 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.951002 (+ 310us) server_negotiation.cc:407] Connection header received
0619 02:45:37.951220 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.951221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.951234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.951260 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.951970 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.952765 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.953792 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.954078 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.954119 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.955423 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.955424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.955424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.955432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.957290 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.957631 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.957632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.957633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.957686 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.957953 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.957955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.957955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.958093 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.958188 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.958286 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.958369 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:45:37.958578 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:37.950919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40214 (local address 127.19.113.124:44455)
0619 02:45:37.950958 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:37.950960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:37.951107 (+ 147us) server_negotiation.cc:407] Connection header received
0619 02:45:37.951151 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:37.951152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:37.951164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:37.951226 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:37.952003 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.952859 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.953900 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:37.954208 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:37.954253 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:37.955600 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:37.955602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:37.955602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:37.955609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:37.957237 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.957844 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.957845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.957845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.957896 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:37.958050 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:37.958051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:37.958051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:37.958163 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:37.958248 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:37.958417 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 02:45:37.958496 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:38.937853 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:38.937518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.66:33459)
0619 02:45:38.937707 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:38.937794 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":103,"threads_started":1}
W20260619 02:45:38.938273 27107 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:32985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:38.938335 27305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:38.937868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:38985)
0619 02:45:38.938223 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:38.938292 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":58,"threads_started":1}
W20260619 02:45:38.938479 27243 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:32985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:38.939069 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:38.938767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:46321)
0619 02:45:38.938976 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:38.939033 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":136,"threads_started":1}
W20260619 02:45:38.939306 26970 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:32985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:39.441347 27307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145a8ff47a8741ab8202851bc659808f)
I20260619 02:45:39.441510 27307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } }
I20260619 02:45:39.441578 27308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145a8ff47a8741ab8202851bc659808f)
I20260619 02:45:39.441699 27308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } }
I20260619 02:45:39.441768 27307 leader_election.cc:288] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985), 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124:44455)
I20260619 02:45:39.442098 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.441901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59390)
0619 02:45:39.442006 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.442055 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":44,"threads_started":1}
I20260619 02:45:39.442094 27308 leader_election.cc:288] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985), 2d98ef611cf8467d98094879c3892ab4 (127.19.113.125:44391)
W20260619 02:45:39.442209 26590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:39.442512 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.442006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59402)
0619 02:45:39.442106 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.442473 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":46,"threads_started":1}
I20260619 02:45:39.442616 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.442515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59408)
0619 02:45:39.442559 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.442588 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:45:39.442708 26776 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
W20260619 02:45:39.442705 26590 leader_election.cc:334] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985): Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:39.443199 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.443046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59424)
0619 02:45:39.443150 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.443173 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
W20260619 02:45:39.443264 26776 leader_election.cc:334] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985): Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:39.445711 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.442048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.19.113.124:44455)
0619 02:45:39.442196 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:39.442198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:39.442315 (+ 117us) server_negotiation.cc:407] Connection header received
0619 02:45:39.442415 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:39.442416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:39.442426 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:39.442462 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:39.442683 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:39.443448 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:39.444811 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:39.445359 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:39.445436 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:39.445446 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:39.445597 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 02:45:39.445639 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":42,"threads_started":1}
I20260619 02:45:39.445713 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.441917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:44455 (local address 127.0.0.1:40224)
0619 02:45:39.442257 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.442266 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:39.442342 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:39.442532 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:39.442533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:39.442536 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:39.442598 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:39.442600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:39.443515 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:39.443516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:39.444721 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:39.444724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:39.445453 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:39.445454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:39.445513 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:39.445525 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:45:39.445592 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:45:39.445640 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":44,"threads_started":1}
I20260619 02:45:39.445806 27315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.442321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.19.113.125:44391)
0619 02:45:39.442546 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:39.442548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:39.442556 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:39.442586 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:39.442586 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:39.442595 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:39.442906 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:39.442943 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:39.443559 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:39.444916 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:39.445463 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:39.445521 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:39.445528 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:39.445680 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 02:45:39.445717 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":98,"thread_start_us":76,"threads_started":1}
I20260619 02:45:39.445875 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.442180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:44391 (local address 127.0.0.1:53674)
0619 02:45:39.442240 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.442249 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:39.442306 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:39.442776 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:39.442777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:39.442779 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:39.442822 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:39.442824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:39.443623 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:39.443624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:39.444842 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:39.444845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:39.445552 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:39.445553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:39.445617 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:39.445620 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:39.445671 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:45:39.445713 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"thread_start_us":29,"threads_started":1}
I20260619 02:45:39.446102 26792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d98ef611cf8467d98094879c3892ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" is_pre_election: true
I20260619 02:45:39.446220 26792 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d98ef611cf8467d98094879c3892ab4 in term 1.
I20260619 02:45:39.446218 26614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d98ef611cf8467d98094879c3892ab4" is_pre_election: true
I20260619 02:45:39.446305 26614 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dfbbbbafe674ac4a6a206ab6a05550b in term 1.
I20260619 02:45:39.446378 26589 leader_election.cc:302] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d98ef611cf8467d98094879c3892ab4, 5dfbbbbafe674ac4a6a206ab6a05550b; no voters: 145a8ff47a8741ab8202851bc659808f
I20260619 02:45:39.446502 27307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:45:39.446507 26777 leader_election.cc:302] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d98ef611cf8467d98094879c3892ab4, 5dfbbbbafe674ac4a6a206ab6a05550b; no voters: 145a8ff47a8741ab8202851bc659808f
I20260619 02:45:39.446542 27307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 145a8ff47a8741ab8202851bc659808f)
I20260619 02:45:39.446561 27307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:45:39.446604 27308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:45:39.446643 27308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 145a8ff47a8741ab8202851bc659808f)
I20260619 02:45:39.446660 27308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:45:39.447428 27307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } }
I20260619 02:45:39.447451 27308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } }
I20260619 02:45:39.447552 27307 leader_election.cc:288] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985), 5dfbbbbafe674ac4a6a206ab6a05550b (127.19.113.124:44455)
I20260619 02:45:39.447564 27308 leader_election.cc:288] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 2 election: Requested vote from peers 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985), 2d98ef611cf8467d98094879c3892ab4 (127.19.113.125:44391)
I20260619 02:45:39.447741 26792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d98ef611cf8467d98094879c3892ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b"
I20260619 02:45:39.447762 26614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d98ef611cf8467d98094879c3892ab4"
I20260619 02:45:39.447824 26792 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d98ef611cf8467d98094879c3892ab4 in current term 2: Already voted for candidate 5dfbbbbafe674ac4a6a206ab6a05550b in this term.
I20260619 02:45:39.447763 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.447636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59432)
0619 02:45:39.447713 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.447738 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 02:45:39.447870 26614 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5dfbbbbafe674ac4a6a206ab6a05550b in current term 2: Already voted for candidate 2d98ef611cf8467d98094879c3892ab4 in this term.
I20260619 02:45:39.447958 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.447655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59436)
0619 02:45:39.447790 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.447815 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260619 02:45:39.448177 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.448090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59440)
0619 02:45:39.448123 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.448153 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260619 02:45:39.448261 26590 leader_election.cc:334] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985): Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:39.448277 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.448203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59450)
0619 02:45:39.448231 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.448256 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260619 02:45:39.448307 26590 leader_election.cc:302] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d98ef611cf8467d98094879c3892ab4; no voters: 145a8ff47a8741ab8202851bc659808f, 5dfbbbbafe674ac4a6a206ab6a05550b
W20260619 02:45:39.448345 26776 leader_election.cc:334] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985): Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:39.448369 27307 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 02:45:39.448387 26776 leader_election.cc:302] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5dfbbbbafe674ac4a6a206ab6a05550b; no voters: 145a8ff47a8741ab8202851bc659808f, 2d98ef611cf8467d98094879c3892ab4
I20260619 02:45:39.448438 27308 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 02:45:39.939270 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.938718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.66:41651)
0619 02:45:39.939169 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.939215 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":81,"threads_started":1}
I20260619 02:45:39.939313 27319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.938977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:41709)
0619 02:45:39.939220 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.939265 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":54,"threads_started":1}
I20260619 02:45:39.939803 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:39.939634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:46691)
0619 02:45:39.939735 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:39.939767 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":49,"threads_started":1}
I20260619 02:45:40.940141 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.939859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.66:35825)
0619 02:45:40.939984 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.940085 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260619 02:45:40.940193 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.939879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:45279)
0619 02:45:40.940106 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.940146 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":54,"threads_started":1}
I20260619 02:45:40.940467 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.940248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:60971)
0619 02:45:40.940401 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.940435 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260619 02:45:40.949860 27324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:40.949962 27324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } }
I20260619 02:45:40.950133 27324 leader_election.cc:288] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985), 2d98ef611cf8467d98094879c3892ab4 (127.19.113.125:44391)
I20260619 02:45:40.950394 26614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d98ef611cf8467d98094879c3892ab4" is_pre_election: true
I20260619 02:45:40.950493 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.950248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59662)
0619 02:45:40.950388 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.950432 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":99,"threads_started":1}
I20260619 02:45:40.950495 26614 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dfbbbbafe674ac4a6a206ab6a05550b in term 2.
I20260619 02:45:40.950644 26777 leader_election.cc:302] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d98ef611cf8467d98094879c3892ab4, 5dfbbbbafe674ac4a6a206ab6a05550b; no voters:
I20260619 02:45:40.950754 27324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 02:45:40.950791 27324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:40.950827 27324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:45:40.950920 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.950808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59668)
0619 02:45:40.950845 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.950895 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 02:45:40.950994 26776 leader_election.cc:334] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985): Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:40.951676 27324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } }
I20260619 02:45:40.951861 27324 leader_election.cc:288] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 3 election: Requested vote from peers 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985), 2d98ef611cf8467d98094879c3892ab4 (127.19.113.125:44391)
I20260619 02:45:40.951975 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.951863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59682)
0619 02:45:40.951913 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.951945 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:45:40.951978 26614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d98ef611cf8467d98094879c3892ab4"
I20260619 02:45:40.952051 26614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:45:40.952265 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.952187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59690)
0619 02:45:40.952222 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.952244 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 02:45:40.952332 26776 leader_election.cc:334] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985): Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
I20260619 02:45:40.952662 26614 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dfbbbbafe674ac4a6a206ab6a05550b in term 3.
I20260619 02:45:40.952814 26777 leader_election.cc:302] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d98ef611cf8467d98094879c3892ab4, 5dfbbbbafe674ac4a6a206ab6a05550b; no voters: 145a8ff47a8741ab8202851bc659808f
I20260619 02:45:40.952898 27324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:45:40.953006 27324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [term 3 LEADER]: Becoming Leader. State: Replica: 5dfbbbbafe674ac4a6a206ab6a05550b, State: Running, Role: LEADER
I20260619 02:45:40.953076 27324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } }
I20260619 02:45:40.953389 27327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dfbbbbafe674ac4a6a206ab6a05550b. Latest consensus state: current_term: 3 leader_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:40.953474 27327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:40.953630 27329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:40.953992 27329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:40.954044 27329 catalog_manager.cc:1259] Loaded cluster ID: 8a5f132352ea4bffa4599117c6d37317
I20260619 02:45:40.954066 27329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:40.954201 27329 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:40.954250 27329 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Loaded TSK: 0
I20260619 02:45:40.954353 27329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:40.961488 26790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:40.963033 26790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:40.964025 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.963935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59702)
0619 02:45:40.963969 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.963997 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 02:45:40.964020 26614 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [term 3 FOLLOWER]: Refusing update from remote peer 5dfbbbbafe674ac4a6a206ab6a05550b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 02:45:40.964352 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.964257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.0.0.1:59716)
0619 02:45:40.964285 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.964315 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260619 02:45:40.964305 27324 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260619 02:45:40.964435 26776 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b -> Peer 145a8ff47a8741ab8202851bc659808f (127.19.113.126:32985): Couldn't send request to peer 145a8ff47a8741ab8202851bc659808f. Status: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:45:40.964865 27331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dfbbbbafe674ac4a6a206ab6a05550b. Latest consensus state: current_term: 3 leader_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:40.964953 27331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:40.965446 27327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:40.965530 27327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:40.965624 27331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "145a8ff47a8741ab8202851bc659808f" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32985 } } peers { permanent_uuid: "2d98ef611cf8467d98094879c3892ab4" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 44391 } attrs { promote: false } } peers { permanent_uuid: "5dfbbbbafe674ac4a6a206ab6a05550b" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 44455 } attrs { promote: false } } }
I20260619 02:45:40.965700 27331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d98ef611cf8467d98094879c3892ab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:40.970010 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.967011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:43431 (local address 127.0.0.1:56168)
0619 02:45:40.967041 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.967051 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:40.967100 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:40.967435 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:40.967436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:40.967438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:40.967481 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:40.967483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.968198 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.968199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:40.969080 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:40.969082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.969790 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.969791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:40.969846 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:40.969849 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:40.969897 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:45:40.969942 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 02:45:40.970012 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.967081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56168 (local address 127.19.113.65:43431)
0619 02:45:40.967302 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:40.967304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:40.967314 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:40.967339 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:40.967340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:40.967351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:40.967409 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:40.967549 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.968127 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.969167 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.969714 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.969787 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:40.969808 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:40.969901 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:45:40.969939 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":44,"threads_started":1}
I20260619 02:45:40.970366 26903 tablet_service.cc:1467] Processing CreateTablet for tablet b335212df13b4b40b494132ef9c37008 (DEFAULT_TABLE table=test-table [id=2bde211a0787424cbc6d157ce6b0a4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:40.970516 26903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b335212df13b4b40b494132ef9c37008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:40.970702 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.967090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:42895 (local address 127.0.0.1:45106)
0619 02:45:40.967463 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.967471 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:40.967513 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:40.967674 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:40.967676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:40.967678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:40.967723 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:40.967725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.968629 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.968630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:40.969549 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:40.969551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.970418 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.970419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:40.970515 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:40.970519 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:40.970598 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:45:40.970629 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":76,"threads_started":1}
I20260619 02:45:40.970719 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.967222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45106 (local address 127.19.113.66:42895)
0619 02:45:40.967320 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:40.967322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:40.967519 (+ 197us) server_negotiation.cc:407] Connection header received
0619 02:45:40.967552 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:40.967553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:40.967564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:40.967631 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:40.967791 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.968542 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.969644 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.970342 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.970409 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:40.970421 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:40.970617 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 02:45:40.970659 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
I20260619 02:45:40.970765 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.967269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.19.113.67:40713)
0619 02:45:40.967368 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:40.967370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:40.967663 (+ 293us) server_negotiation.cc:407] Connection header received
0619 02:45:40.967690 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:40.967691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:40.967701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:40.967760 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:40.967915 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.968635 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.969814 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.970377 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.970435 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:40.970447 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:40.970656 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:45:40.970701 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":49,"threads_started":1}
I20260619 02:45:40.971002 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.967096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:40713 (local address 127.0.0.1:48994)
0619 02:45:40.967575 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.967582 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:40.967620 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:40.967812 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:40.967812 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:40.967814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:40.967849 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:40.967851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.968705 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.968706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:40.969722 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:40.969724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:40.970473 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:40.970474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:40.970524 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:40.970527 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:40.970566 (+ 39us) client_negotiation.cc:240] Negotiation successful
0619 02:45:40.970606 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":82,"threads_started":1}
I20260619 02:45:40.971249 27176 tablet_service.cc:1467] Processing CreateTablet for tablet b335212df13b4b40b494132ef9c37008 (DEFAULT_TABLE table=test-table [id=2bde211a0787424cbc6d157ce6b0a4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:40.971374 27176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b335212df13b4b40b494132ef9c37008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:40.972105 27040 tablet_service.cc:1467] Processing CreateTablet for tablet b335212df13b4b40b494132ef9c37008 (DEFAULT_TABLE table=test-table [id=2bde211a0787424cbc6d157ce6b0a4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:40.972151 27341 tablet_bootstrap.cc:492] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7: Bootstrap starting.
I20260619 02:45:40.972227 27040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b335212df13b4b40b494132ef9c37008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:40.972527 27341 tablet_bootstrap.cc:654] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:40.972810 27342 tablet_bootstrap.cc:492] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: Bootstrap starting.
I20260619 02:45:40.973044 27341 tablet_bootstrap.cc:492] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7: No bootstrap required, opened a new log
I20260619 02:45:40.973078 27341 ts_tablet_manager.cc:1397] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:40.973158 27341 raft_consensus.cc:348] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:40.973203 27341 raft_consensus.cc:374] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:40.973214 27341 raft_consensus.cc:729] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579681295fbc400ea66bae3b061cbfa7, State: Initialized, Role: FOLLOWER
I20260619 02:45:40.973250 27341 consensus_queue.cc:260] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:40.973232 27342 tablet_bootstrap.cc:654] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:40.973399 27341 ts_tablet_manager.cc:1428] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:40.973764 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.973620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:56513)
0619 02:45:40.973701 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.973736 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 02:45:40.973985 26969 heartbeater.cc:503] Master 127.19.113.124:44455 was elected leader, sending a full tablet report...
I20260619 02:45:40.974326 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.974242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:37767)
0619 02:45:40.974280 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.974303 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:45:40.974344 27346 tablet_bootstrap.cc:492] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: Bootstrap starting.
I20260619 02:45:40.974551 27342 tablet_bootstrap.cc:492] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: No bootstrap required, opened a new log
I20260619 02:45:40.974608 27342 ts_tablet_manager.cc:1397] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:40.974713 27342 raft_consensus.cc:348] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:40.974777 27342 raft_consensus.cc:374] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:40.974797 27342 raft_consensus.cc:729] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfcce9bce66242dd9e4df6ebf3a29ff6, State: Initialized, Role: FOLLOWER
I20260619 02:45:40.974841 27342 consensus_queue.cc:260] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:40.974890 27346 tablet_bootstrap.cc:654] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:40.975029 27342 ts_tablet_manager.cc:1428] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:40.975275 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.975184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:58997)
0619 02:45:40.975223 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.975252 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 02:45:40.975502 27346 tablet_bootstrap.cc:492] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: No bootstrap required, opened a new log
I20260619 02:45:40.975528 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.975461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:47855)
0619 02:45:40.975489 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.975512 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 02:45:40.975553 27346 ts_tablet_manager.cc:1397] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:40.975669 27346 raft_consensus.cc:348] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:40.975723 27242 heartbeater.cc:503] Master 127.19.113.124:44455 was elected leader, sending a full tablet report...
I20260619 02:45:40.975739 27346 raft_consensus.cc:374] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:40.975757 27346 raft_consensus.cc:729] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3dcb4e39e0428c8ca53257cf94b645, State: Initialized, Role: FOLLOWER
I20260619 02:45:40.975797 27346 consensus_queue.cc:260] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:40.975970 27346 ts_tablet_manager.cc:1428] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:40.976270 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:40.976191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.66:41829)
0619 02:45:40.976219 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:40.976245 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260619 02:45:40.976509 27106 heartbeater.cc:503] Master 127.19.113.124:44455 was elected leader, sending a full tablet report...
I20260619 02:45:40.997006 27344 raft_consensus.cc:482] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:40.997099 27344 raft_consensus.cc:504] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:40.997328 27344 leader_election.cc:288] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713), ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895)
I20260619 02:45:40.997476 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b335212df13b4b40b494132ef9c37008" candidate_uuid: "579681295fbc400ea66bae3b061cbfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" is_pre_election: true
I20260619 02:45:40.997571 27196 raft_consensus.cc:2454] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579681295fbc400ea66bae3b061cbfa7 in term 0.
I20260619 02:45:40.997560 27060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b335212df13b4b40b494132ef9c37008" candidate_uuid: "579681295fbc400ea66bae3b061cbfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" is_pre_election: true
I20260619 02:45:40.997668 27060 raft_consensus.cc:2454] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579681295fbc400ea66bae3b061cbfa7 in term 0.
I20260619 02:45:40.997726 26856 leader_election.cc:302] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579681295fbc400ea66bae3b061cbfa7, dfcce9bce66242dd9e4df6ebf3a29ff6; no voters:
I20260619 02:45:40.997833 27344 raft_consensus.cc:2781] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:40.997867 27344 raft_consensus.cc:482] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:40.997886 27344 raft_consensus.cc:3037] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:40.998575 27344 raft_consensus.cc:504] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:40.998694 27344 leader_election.cc:288] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [CANDIDATE]: Term 1 election: Requested vote from peers dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713), ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895)
I20260619 02:45:40.998908 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b335212df13b4b40b494132ef9c37008" candidate_uuid: "579681295fbc400ea66bae3b061cbfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6"
I20260619 02:45:40.998919 27060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b335212df13b4b40b494132ef9c37008" candidate_uuid: "579681295fbc400ea66bae3b061cbfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3dcb4e39e0428c8ca53257cf94b645"
I20260619 02:45:40.998982 27196 raft_consensus.cc:3037] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:40.999002 27060 raft_consensus.cc:3037] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:40.999553 27196 raft_consensus.cc:2454] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579681295fbc400ea66bae3b061cbfa7 in term 1.
I20260619 02:45:40.999559 27060 raft_consensus.cc:2454] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579681295fbc400ea66bae3b061cbfa7 in term 1.
I20260619 02:45:40.999714 26856 leader_election.cc:302] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 579681295fbc400ea66bae3b061cbfa7, dfcce9bce66242dd9e4df6ebf3a29ff6, ec3dcb4e39e0428c8ca53257cf94b645; no voters:
I20260619 02:45:40.999827 27344 raft_consensus.cc:2781] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:40.999989 27344 raft_consensus.cc:686] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [term 1 LEADER]: Becoming Leader. State: Replica: 579681295fbc400ea66bae3b061cbfa7, State: Running, Role: LEADER
I20260619 02:45:41.000038 27344 consensus_queue.cc:237] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } }
I20260619 02:45:41.000586 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.000457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:52951)
0619 02:45:41.000526 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:41.000557 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 02:45:41.000834 26790 catalog_manager.cc:5495] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 579681295fbc400ea66bae3b061cbfa7 (127.19.113.65). New cstate: current_term: 1 leader_uuid: "579681295fbc400ea66bae3b061cbfa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579681295fbc400ea66bae3b061cbfa7" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 43431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 } health_report { overall_health: UNKNOWN } } }
I20260619 02:45:41.005573 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.003640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.19.113.65:43431)
0619 02:45:41.003682 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:41.003684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:41.003709 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:45:41.003764 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:41.003765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:41.003773 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:41.003814 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:41.003904 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:41.004367 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:41.005061 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:41.005247 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:41.005297 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:41.005376 (+ 79us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:41.005401 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:41.005432 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:41.005494 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:45:41.005510 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:41.006325 27060 raft_consensus.cc:1261] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 1 FOLLOWER]: Refusing update from remote peer 579681295fbc400ea66bae3b061cbfa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:41.006345 27196 raft_consensus.cc:1261] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 1 FOLLOWER]: Refusing update from remote peer 579681295fbc400ea66bae3b061cbfa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:41.006598 27344 consensus_queue.cc:1035] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3dcb4e39e0428c8ca53257cf94b645" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:41.006729 27350 consensus_queue.cc:1035] T b335212df13b4b40b494132ef9c37008 P 579681295fbc400ea66bae3b061cbfa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfcce9bce66242dd9e4df6ebf3a29ff6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 40713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:41.007835 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.007729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.66:45551)
0619 02:45:41.007768 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:41.007802 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 02:45:41.008255 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.008135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:39019)
0619 02:45:41.008202 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:41.008227 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 02:45:41.008699 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.008599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:57261)
0619 02:45:41.008635 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:41.008669 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 02:45:41.009001 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.008911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.67:53805)
0619 02:45:41.008943 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:41.008965 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 02:45:41.009294 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.009210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.66:33701)
0619 02:45:41.009244 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:41.009267 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 02:45:41.009565 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.009457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32985 (local address 127.19.113.65:34765)
0619 02:45:41.009512 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:41.009540 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:45:41.009990 26790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:41.010071 26790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:41.011238 26790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dfbbbbafe674ac4a6a206ab6a05550b: Sending DeleteTablet for 3 replicas of tablet b335212df13b4b40b494132ef9c37008
I20260619 02:45:41.011507 27040 tablet_service.cc:1514] Processing DeleteTablet for tablet b335212df13b4b40b494132ef9c37008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:41 UTC) from {username='oryx', principal='oryx/127.19.113.124@KRBTEST.COM'} at 127.0.0.1:45106
I20260619 02:45:41.011556 27176 tablet_service.cc:1514] Processing DeleteTablet for tablet b335212df13b4b40b494132ef9c37008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:41 UTC) from {username='oryx', principal='oryx/127.19.113.124@KRBTEST.COM'} at 127.0.0.1:48994
I20260619 02:45:41.011575 26903 tablet_service.cc:1514] Processing DeleteTablet for tablet b335212df13b4b40b494132ef9c37008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:41 UTC) from {username='oryx', principal='oryx/127.19.113.124@KRBTEST.COM'} at 127.0.0.1:56168
I20260619 02:45:41.011713 27363 tablet_replica.cc:330] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: stopping tablet replica
I20260619 02:45:41.011768 27363 raft_consensus.cc:2229] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:41.011809 27363 raft_consensus.cc:2258] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:41.011905 27362 tablet_replica.cc:330] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: stopping tablet replica
I20260619 02:45:41.011955 27362 raft_consensus.cc:2229] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:41.011997 27362 raft_consensus.cc:2258] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:41.012037 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 26751
I20260619 02:45:41.012286 27363 ts_tablet_manager.cc:1905] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:41.013237 27362 ts_tablet_manager.cc:1905] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:41.013729 27363 ts_tablet_manager.cc:1918] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:41.013778 27363 log.cc:1196] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/b335212df13b4b40b494132ef9c37008
I20260619 02:45:41.013968 27363 ts_tablet_manager.cc:1939] T b335212df13b4b40b494132ef9c37008 P dfcce9bce66242dd9e4df6ebf3a29ff6: Deleting consensus metadata
I20260619 02:45:41.014245 26774 catalog_manager.cc:4841] TS dfcce9bce66242dd9e4df6ebf3a29ff6 (127.19.113.67:40713): tablet b335212df13b4b40b494132ef9c37008 (table test-table [id=2bde211a0787424cbc6d157ce6b0a4dc]) successfully deleted
I20260619 02:45:41.014688 27362 ts_tablet_manager.cc:1918] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:41.014742 27362 log.cc:1196] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/b335212df13b4b40b494132ef9c37008
I20260619 02:45:41.014997 27362 ts_tablet_manager.cc:1939] T b335212df13b4b40b494132ef9c37008 P ec3dcb4e39e0428c8ca53257cf94b645: Deleting consensus metadata
I20260619 02:45:41.015261 26776 catalog_manager.cc:4841] TS ec3dcb4e39e0428c8ca53257cf94b645 (127.19.113.66:42895): tablet b335212df13b4b40b494132ef9c37008 (table test-table [id=2bde211a0787424cbc6d157ce6b0a4dc]) successfully deleted
I20260619 02:45:41.018664 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:41.018555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:43431 (local address 127.0.0.1:56172)
0619 02:45:41.018601 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:41.018628 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:43431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 02:45:41.018770 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 26977
W20260619 02:45:41.018813 26774 catalog_manager.cc:4584] TS 579681295fbc400ea66bae3b061cbfa7 (127.19.113.65:43431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b335212df13b4b40b494132ef9c37008: Network error: Client connection negotiation failed: client connection to 127.19.113.65:43431: connect: Connection refused (error 111)
I20260619 02:45:41.023283 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 27113
I20260619 02:45:41.027530 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 26470
I20260619 02:45:41.031662 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 26658
2026-06-19T02:45:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5829 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:41 dist-test-slave-zqpb krb5kdc[27367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:41 dist-test-slave-zqpb krb5kdc[27367](info): set up 2 sockets
Jun 19 02:45:41 dist-test-slave-zqpb krb5kdc[27367](info): commencing operation
krb5kdc: starting...
W20260619 02:45:41.991165 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.126.keytab.
Entry for principal oryx/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.126.keytab.
I20260619 02:45:42.101523 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:33279
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.126.keytab
--principal=oryx/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:33279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:42.182435 27380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:42.182653 27380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:42.182683 27380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:42.183917 27380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:42.183964 27380 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:45:42.183980 27380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:42.183997 27380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:42.184013 27380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:42.184031 27380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:42.185577 27380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:33279
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.126.keytab
--principal=oryx/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:33279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:42.185838 27380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:42.186053 27380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:42.190671 27388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:42.190671 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:42.190708 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:42.190959 27380 server_base.cc:1034] running on GCE node
I20260619 02:45:42.191174 27380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:42.191450 27380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:42.192590 27380 hybrid_clock.cc:648] HybridClock initialized: now 1781837142192580 us; error 34 us; skew 500 ppm
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:42.195029 27380 init.cc:377] Logged in from keytab as oryx/127.19.113.126@KRBTEST.COM (short username oryx)
I20260619 02:45:42.195472 27380 webserver.cc:460] Webserver started at http://127.19.113.126:46679/ using document root <none> and password file <none>
I20260619 02:45:42.195672 27380 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:42.195714 27380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:42.195827 27380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:42.196900 27380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "d8b4353607844288b56c1c31128f79b2"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "10caf8657763effd9c9ad4e8882f88cd"
server_key_iv: "efe48c5bc5771013e2cfc8c5c48cb61f"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.197227 27380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "d8b4353607844288b56c1c31128f79b2"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "10caf8657763effd9c9ad4e8882f88cd"
server_key_iv: "efe48c5bc5771013e2cfc8c5c48cb61f"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.198546 27380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:42.199357 27395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.199502 27380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.199584 27380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "d8b4353607844288b56c1c31128f79b2"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "10caf8657763effd9c9ad4e8882f88cd"
server_key_iv: "efe48c5bc5771013e2cfc8c5c48cb61f"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.199653 27380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:42.217384 27380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:42.219426 27380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:42.219624 27380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:42.224207 27380 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:33279
I20260619 02:45:42.224264 27457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:33279 every 8 connection(s)
I20260619 02:45:42.224593 27380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:42.225270 27458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:42.226279 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 27380
I20260619 02:45:42.226397 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:42.226637 19909 external_mini_cluster.cc:1428] Setting key 3ae0d24f5d49c5d7b6b0fec2a205a2e7
I20260619 02:45:42.228078 27458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: Bootstrap starting.
I20260619 02:45:42.228683 27458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:42.228979 27458 log.cc:824] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:42.229640 27458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: No bootstrap required, opened a new log
I20260619 02:45:42.231552 27458 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b4353607844288b56c1c31128f79b2" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33279 } }
I20260619 02:45:42.231731 27458 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:42.231781 27458 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b4353607844288b56c1c31128f79b2, State: Initialized, Role: FOLLOWER
I20260619 02:45:42.231914 27458 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b4353607844288b56c1c31128f79b2" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33279 } }
I20260619 02:45:42.231989 27458 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:42.232019 27458 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:42.232066 27458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:42.232585 27458 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b4353607844288b56c1c31128f79b2" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33279 } }
I20260619 02:45:42.232681 27458 leader_election.cc:302] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8b4353607844288b56c1c31128f79b2; no voters:
I20260619 02:45:42.232848 27458 leader_election.cc:288] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:42.232906 27463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:42.233038 27463 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [term 1 LEADER]: Becoming Leader. State: Replica: d8b4353607844288b56c1c31128f79b2, State: Running, Role: LEADER
I20260619 02:45:42.233119 27458 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:42.233119 27463 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b4353607844288b56c1c31128f79b2" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33279 } }
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:42.233515 27464 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8b4353607844288b56c1c31128f79b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b4353607844288b56c1c31128f79b2" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33279 } } }
I20260619 02:45:42.233600 27464 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:42.233762 27465 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8b4353607844288b56c1c31128f79b2. Latest consensus state: current_term: 1 leader_uuid: "d8b4353607844288b56c1c31128f79b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b4353607844288b56c1c31128f79b2" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33279 } } }
I20260619 02:45:42.233878 27465 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:42.234004 27469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:42.234513 27469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:42.237372 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.227203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.19.113.126:33279)
0619 02:45:42.227381 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.227389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.227414 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:45:42.227939 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.227946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.228001 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.228125 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.228748 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.229938 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.230660 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.231137 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.231232 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.233872 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:42.233877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:42.233878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:42.233891 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:42.235864 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.236202 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.236205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.236205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.236307 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.236482 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.236483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.236483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.236792 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:42.236925 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.237021 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.237157 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260619 02:45:42.237960 27480 mvcc.cc:205] Tried to move back new op lower bound from 7298404934596222976 to 7298404934587539456. Current Snapshot: MvccSnapshot[applied={T|T < 7298404934596222976}]
I20260619 02:45:42.238525 27469 catalog_manager.cc:1347] Generated new cluster ID: 6d7b2650cf3d438d9b8598701977361c
I20260619 02:45:42.238570 27469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:42.251200 27469 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:42.251813 27469 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:42.254117 27469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: Generated new TSK 0
I20260619 02:45:42.254278 27469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab.
Entry for principal oryx/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab.
I20260619 02:45:42.281205 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33279
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab
--principal=oryx/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:42.361547 27484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:42.361753 27484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:42.361776 27484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:42.363008 27484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:42.363051 27484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:42.363071 27484 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:42.363090 27484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:42.364946 27484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab
--principal=oryx/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:42.365173 27484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:42.365378 27484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:42.370087 27490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:42.370105 27489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:42.370100 27492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:42.370409 27484 server_base.cc:1034] running on GCE node
I20260619 02:45:42.370625 27484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:42.370823 27484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:42.371979 27484 hybrid_clock.cc:648] HybridClock initialized: now 1781837142371963 us; error 26 us; skew 500 ppm
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:42.374313 27484 init.cc:377] Logged in from keytab as oryx/127.19.113.65@KRBTEST.COM (short username oryx)
I20260619 02:45:42.374696 27484 webserver.cc:460] Webserver started at http://127.19.113.65:44621/ using document root <none> and password file <none>
I20260619 02:45:42.374922 27484 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:42.374979 27484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:42.375073 27484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:42.376096 27484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "2191278f82f447f3a8c08f1813fd5c68"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "657a60664cf7613cca5d8bb521435e8c"
server_key_iv: "4ae131944c598c263d632f4a7cb1f033"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.376384 27484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "2191278f82f447f3a8c08f1813fd5c68"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "657a60664cf7613cca5d8bb521435e8c"
server_key_iv: "4ae131944c598c263d632f4a7cb1f033"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.377585 27484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:42.378237 27499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.378388 27484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:42.378470 27484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "2191278f82f447f3a8c08f1813fd5c68"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "657a60664cf7613cca5d8bb521435e8c"
server_key_iv: "4ae131944c598c263d632f4a7cb1f033"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.378531 27484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:42.391103 27484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:42.393059 27484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:42.393196 27484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:42.393684 27484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:42.393726 27484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.393756 27484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:42.393774 27484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.399978 27484 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:39671
I20260619 02:45:42.400034 27614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:39671 every 8 connection(s)
I20260619 02:45:42.400331 27484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:42.403466 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.394171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:55377 (local address 127.19.113.126:33279)
0619 02:45:42.394215 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.394217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.394451 (+ 234us) server_negotiation.cc:407] Connection header received
0619 02:45:42.395032 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.395033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.395046 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.395091 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.395892 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.396718 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.398200 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.398443 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.398499 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.400972 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:42.400973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:42.400974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:42.400982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:42.402072 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.402499 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.402500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.402501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.402560 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.402771 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.402771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.402772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.402939 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:42.402996 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.403259 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.403375 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:45:42.403602 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.394168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33279 (local address 127.19.113.65:55377)
0619 02:45:42.394335 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.394371 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.394925 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.395162 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.395166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.395524 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:42.395806 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.395812 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.396810 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.396811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.398121 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.398125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.398519 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.398521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.398669 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.399273 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:42.399281 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:42.400880 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:42.402148 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.402155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.402157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.402435 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.402612 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.402613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.402613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.402717 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.403002 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:42.403009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:42.403142 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.403227 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.403394 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260619 02:45:42.404086 27615 heartbeater.cc:347] Connected to a master server at 127.19.113.126:33279
I20260619 02:45:42.404165 27615 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:42.404352 27615 heartbeater.cc:511] Master 127.19.113.126:33279 requested a full tablet report, sending...
I20260619 02:45:42.404861 27411 ts_manager.cc:194] Registered new tserver with Master: 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671)
I20260619 02:45:42.405637 27411 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:55377
I20260619 02:45:42.406385 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 27484
I20260619 02:45:42.406476 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:42.406634 19909 external_mini_cluster.cc:1428] Setting key 4f504a4c66dd4b16e077a19f0b6974a6
I20260619 02:45:42.410595 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.405050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33279 (local address 127.19.113.65:59503)
0619 02:45:42.405157 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.405167 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.405217 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.405325 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.405326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.405469 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:42.405517 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.405518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.406333 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.406334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.407048 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.407051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.407449 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.407450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.407518 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.407936 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:42.407941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:42.408249 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:42.409745 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.409746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.409747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.409914 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.410094 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.410094 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.410095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.410144 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.410365 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:42.410365 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:42.410404 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.410452 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.410528 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":59,"threads_started":1}
I20260619 02:45:42.410637 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.405112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:59503 (local address 127.19.113.126:33279)
0619 02:45:42.405149 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.405151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.405207 (+ 56us) server_negotiation.cc:407] Connection header received
0619 02:45:42.405252 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.405253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.405261 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.405322 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.405568 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.406239 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.407109 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.407377 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.407434 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.408336 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:42.408337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:42.408337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:42.408343 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:42.409675 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.409973 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.409974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.409975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.410033 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.410190 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.410191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.410191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.410312 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:42.410369 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.410488 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.410557 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for oryx/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab.
Entry for principal oryx/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab.
I20260619 02:45:42.430327 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33279
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab
--principal=oryx/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:42.511627 27624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:42.511852 27624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:42.511876 27624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:42.513015 27624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:42.513049 27624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:42.513068 27624 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:42.513084 27624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:42.514575 27624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab
--principal=oryx/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:42.514770 27624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:42.514993 27624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:42.519634 27629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:42.519742 27624 server_base.cc:1034] running on GCE node
W20260619 02:45:42.519639 27632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:42.519824 27630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:42.520072 27624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:42.520303 27624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:42.521481 27624 hybrid_clock.cc:648] HybridClock initialized: now 1781837142521467 us; error 26 us; skew 500 ppm
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:42.524174 27624 init.cc:377] Logged in from keytab as oryx/127.19.113.66@KRBTEST.COM (short username oryx)
I20260619 02:45:42.524628 27624 webserver.cc:460] Webserver started at http://127.19.113.66:37707/ using document root <none> and password file <none>
I20260619 02:45:42.524832 27624 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:42.524884 27624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:42.525015 27624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:42.526007 27624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "cb459bd0c9564ce790f7a4af4ef683c2"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "8104acb95fda25258bca0267f144c4fd"
server_key_iv: "2ece15786dcfc3b9e90725b4a0a35b74"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.526312 27624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "cb459bd0c9564ce790f7a4af4ef683c2"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "8104acb95fda25258bca0267f144c4fd"
server_key_iv: "2ece15786dcfc3b9e90725b4a0a35b74"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.527608 27624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:42.528301 27639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.528479 27624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:42.528553 27624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "cb459bd0c9564ce790f7a4af4ef683c2"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "8104acb95fda25258bca0267f144c4fd"
server_key_iv: "2ece15786dcfc3b9e90725b4a0a35b74"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.528604 27624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:42.540161 27624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:42.541862 27624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:42.541975 27624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:42.542412 27624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:42.542452 27624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.542485 27624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:42.542505 27624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.548822 27624 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:44827
I20260619 02:45:42.548900 27754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:44827 every 8 connection(s)
I20260619 02:45:42.549185 27624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:42.552552 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.542994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:47855 (local address 127.19.113.126:33279)
0619 02:45:42.543050 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.543052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.543265 (+ 213us) server_negotiation.cc:407] Connection header received
0619 02:45:42.543781 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.543783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.543798 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.543866 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.544788 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.545550 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.547037 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.547323 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.547393 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.549889 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:42.549891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:42.549891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:42.549899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:42.551245 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.551618 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.551619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.551619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.551686 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.551935 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.551936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.551936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.552064 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:42.552125 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.552419 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.552485 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:45:42.552762 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.542977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33279 (local address 127.19.113.66:47855)
0619 02:45:42.543134 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.543171 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.543676 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.543953 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.543957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.544318 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:42.544686 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.544692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.545647 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.545648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.546935 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.546938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.547422 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.547423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.547535 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.548158 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:42.548170 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:42.549795 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:42.551314 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.551320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.551322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.551557 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.551732 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.551733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.551733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.551837 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.552153 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:42.552159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:42.552297 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.552400 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.552531 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260619 02:45:42.553282 27755 heartbeater.cc:347] Connected to a master server at 127.19.113.126:33279
I20260619 02:45:42.553386 27755 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:42.553555 27755 heartbeater.cc:511] Master 127.19.113.126:33279 requested a full tablet report, sending...
I20260619 02:45:42.553972 27412 ts_manager.cc:194] Registered new tserver with Master: cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827)
I20260619 02:45:42.554473 27412 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:47855
I20260619 02:45:42.555344 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 27624
I20260619 02:45:42.555445 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:42.555634 19909 external_mini_cluster.cc:1428] Setting key ab2e869375f00f0fa1e0284ddb6eeed7
I20260619 02:45:42.559475 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.554223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33279 (local address 127.19.113.66:47543)
0619 02:45:42.554352 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.554384 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.554446 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.554566 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.554568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.554769 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:42.554830 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.554832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.555502 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.555503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.556123 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.556125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.556460 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.556461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.556524 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.556920 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:42.556925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:42.557204 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:42.558560 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.558561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.558561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.558717 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.558909 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.558909 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.558910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.558967 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.559238 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:42.559239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:42.559274 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.559317 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.559398 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260619 02:45:42.559510 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.554296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:47543 (local address 127.19.113.126:33279)
0619 02:45:42.554337 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.554338 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.554431 (+ 93us) server_negotiation.cc:407] Connection header received
0619 02:45:42.554496 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.554497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.554504 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.554551 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.554928 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.555409 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.556188 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.556404 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.556454 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.557280 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:42.557281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:42.557281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:42.557288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:42.558477 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.558773 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.558774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.558774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.558832 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.559013 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.559013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.559014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.559185 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:42.559250 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.559349 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.559427 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for oryx/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab.
Entry for principal oryx/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab.
I20260619 02:45:42.583225 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:33279
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab
--principal=oryx/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:42.664516 27764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:42.664723 27764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:42.664757 27764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:42.665941 27764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:42.665987 27764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:42.666016 27764 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:45:42.666039 27764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:42.667599 27764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab
--principal=oryx/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.113.126:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:42.667843 27764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:42.668068 27764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:42.672796 27770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:42.672959 27764 server_base.cc:1034] running on GCE node
W20260619 02:45:42.672886 27769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:42.672797 27772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:42.673290 27764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:42.673528 27764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:42.674677 27764 hybrid_clock.cc:648] HybridClock initialized: now 1781837142674664 us; error 26 us; skew 500 ppm
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:42.677139 27764 init.cc:377] Logged in from keytab as oryx/127.19.113.67@KRBTEST.COM (short username oryx)
I20260619 02:45:42.677551 27764 webserver.cc:460] Webserver started at http://127.19.113.67:33595/ using document root <none> and password file <none>
I20260619 02:45:42.677755 27764 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:42.677802 27764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:42.677966 27764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:42.679046 27764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "7c77bb1920234a3aad0959cf780b26d9"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "f50661835d26cd5b391475e4e2ef12e9"
server_key_iv: "c4afc9360bfc8385eedc397b76fcfb3f"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.679360 27764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "7c77bb1920234a3aad0959cf780b26d9"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "f50661835d26cd5b391475e4e2ef12e9"
server_key_iv: "c4afc9360bfc8385eedc397b76fcfb3f"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.680549 27764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:42.681236 27779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.681391 27764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:42.681476 27764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "7c77bb1920234a3aad0959cf780b26d9"
format_stamp: "Formatted at 2026-06-19 02:45:42 on dist-test-slave-zqpb"
server_key: "f50661835d26cd5b391475e4e2ef12e9"
server_key_iv: "c4afc9360bfc8385eedc397b76fcfb3f"
server_key_version: "encryptionkey@0"
I20260619 02:45:42.681548 27764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:42.712198 27764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:42.714043 27764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:42.714175 27764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:42.714668 27764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:42.714706 27764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.714735 27764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:42.714751 27764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:42.721130 27764 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:41383
I20260619 02:45:42.721194 27894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:41383 every 8 connection(s)
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:42.721509 27764 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:42.724292 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.715107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:60219 (local address 127.19.113.126:33279)
0619 02:45:42.715154 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.715156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.715376 (+ 220us) server_negotiation.cc:407] Connection header received
0619 02:45:42.715910 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.715911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.715923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.715986 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.716784 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.717544 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.719053 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.719348 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.719407 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.721961 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:42.721962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:42.721963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:42.721971 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:42.723084 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.723507 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.723508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.723509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.723559 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.723781 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.723781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.723781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.723881 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:42.723932 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.724163 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.724227 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:45:42.724449 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.715113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33279 (local address 127.19.113.67:60219)
0619 02:45:42.715253 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.715300 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.715817 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.716066 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.716071 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.716418 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:42.716704 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.716710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.717638 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.717639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.718965 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.718970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.719435 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.719436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.719583 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.720165 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:42.720176 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:42.721870 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:42.723181 (+ 1311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.723188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.723190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.723441 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.723625 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.723626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.723626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.723730 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.723925 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:42.723928 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:42.724049 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.724129 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.724251 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260619 02:45:42.724895 27895 heartbeater.cc:347] Connected to a master server at 127.19.113.126:33279
I20260619 02:45:42.725001 27895 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:42.725172 27895 heartbeater.cc:511] Master 127.19.113.126:33279 requested a full tablet report, sending...
I20260619 02:45:42.725569 27412 ts_manager.cc:194] Registered new tserver with Master: 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383)
I20260619 02:45:42.726043 27412 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:60219
I20260619 02:45:42.728312 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 27764
I20260619 02:45:42.728428 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:42.728668 19909 external_mini_cluster.cc:1428] Setting key df2c4ba9770ce771133e5fcec8c538c3
I20260619 02:45:42.729562 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:45:42.731478 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.725944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:58769 (local address 127.19.113.126:33279)
0619 02:45:42.725998 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.726000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.726025 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:45:42.726090 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.726091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.726100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.726154 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.726520 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.727049 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.727702 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.727898 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.727949 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.728876 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:42.728878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:42.728878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:42.728887 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:42.730425 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.730725 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.730726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.730727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.730783 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.730972 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.730973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.730973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.731089 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:42.731152 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.731306 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.731379 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:45:42.731484 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.725862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33279 (local address 127.19.113.67:58769)
0619 02:45:42.725972 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.725982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.726039 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.726197 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.726198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.726394 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:42.726449 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.726451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.727130 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.727130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.727635 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.727636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.727953 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.727953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.728024 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.728409 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:42.728414 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:42.728786 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:42.730500 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.730501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.730501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.730664 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.730843 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:42.730843 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:42.730843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:42.730925 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:42.731146 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:42.731147 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:42.731213 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.731272 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.731359 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":60,"threads_started":1}
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:45:42.750134 27461 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:42.745289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.19.113.126:33279)
0619 02:45:42.745347 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.745349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.745358 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:42.745386 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.745386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.745395 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.745439 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.745937 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.746418 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.747181 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.747360 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.747412 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.750052 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39454: BlockingRecv error: recv got EOF from 127.0.0.1:39454 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 02:45:42 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:42.756852 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.750814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.19.113.126:33279)
0619 02:45:42.750858 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.750859 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.750934 (+ 75us) server_negotiation.cc:407] Connection header received
0619 02:45:42.751023 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.751024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.751032 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.751085 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:42.751463 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.751971 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.752586 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.752779 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.752823 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.754765 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:42.754767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:42.754767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:42.754774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:42.755874 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.756180 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.756181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.756182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.756240 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:42.756422 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:42.756422 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:42.756423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:42.756528 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:42.756615 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.756723 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.756785 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:45:42.758324 27412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:42.760277 27412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:42.766633 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.762472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:44827 (local address 127.0.0.1:59928)
0619 02:45:42.762629 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.762671 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.762735 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.763176 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.763179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.763182 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:42.763259 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.763261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.764252 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.764253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.765448 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.765451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.766361 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.766362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.766450 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.766465 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.766526 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.766568 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":54,"threads_started":1}
I20260619 02:45:42.766698 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.762600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.19.113.66:44827)
0619 02:45:42.762999 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.763004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.763015 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:42.763065 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.763067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.763083 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.763158 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:42.763323 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.764182 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.765531 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.766280 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.766350 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.766429 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.766554 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.766593 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":81,"threads_started":1}
I20260619 02:45:42.767048 27917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.762480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39671 (local address 127.0.0.1:54690)
0619 02:45:42.762991 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.763002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.763066 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.763341 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.763342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.763344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:42.763400 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.763402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.764710 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.764711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.765729 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.765731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.766827 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.766828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.766900 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.766903 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.766945 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.766981 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":90,"threads_started":1}
I20260619 02:45:42.767292 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.762788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.19.113.65:39671)
0619 02:45:42.762994 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.762998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.763054 (+ 56us) server_negotiation.cc:407] Connection header received
0619 02:45:42.763132 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.763134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.763150 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.763228 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:42.763479 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.764612 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.765800 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.766282 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.766345 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.766425 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.767149 (+ 724us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.767197 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
I20260619 02:45:42.767333 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.762672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:41383 (local address 127.0.0.1:45958)
0619 02:45:42.763266 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:42.763275 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:42.763405 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:42.763593 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:42.763594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:42.763596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:42.763651 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.763652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.764840 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.764841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.765804 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:42.765807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.766670 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.766670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:42.767182 (+ 512us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.767186 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:42.767239 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:45:42.767275 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260619 02:45:42.767408 27689 tablet_service.cc:1467] Processing CreateTablet for tablet 631ead3d29494490931b2f3b7c30b530 (DEFAULT_TABLE table=test-table [id=ed1a95395fb94cb6aa6ff46cacda6e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:42.767570 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:42.762788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.19.113.67:41383)
0619 02:45:42.762976 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:42.762981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:42.763381 (+ 400us) server_negotiation.cc:407] Connection header received
0619 02:45:42.763504 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:42.763507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:42.763523 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:42.763592 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:42.763706 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.764742 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.765893 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:42.766591 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:42.766658 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:42.766728 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:42.767418 (+ 690us) server_negotiation.cc:299] Negotiation successful
0619 02:45:42.767475 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260619 02:45:42.767812 27689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 631ead3d29494490931b2f3b7c30b530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:42.770998 27923 tablet_bootstrap.cc:492] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Bootstrap starting.
I20260619 02:45:42.771019 27829 tablet_service.cc:1467] Processing CreateTablet for tablet 631ead3d29494490931b2f3b7c30b530 (DEFAULT_TABLE table=test-table [id=ed1a95395fb94cb6aa6ff46cacda6e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:42.771224 27549 tablet_service.cc:1467] Processing CreateTablet for tablet 631ead3d29494490931b2f3b7c30b530 (DEFAULT_TABLE table=test-table [id=ed1a95395fb94cb6aa6ff46cacda6e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:42.771365 27829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 631ead3d29494490931b2f3b7c30b530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:42.771556 27549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 631ead3d29494490931b2f3b7c30b530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:42.771641 27923 tablet_bootstrap.cc:654] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:42.771956 27923 log.cc:824] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:42.772739 27923 tablet_bootstrap.cc:492] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: No bootstrap required, opened a new log
I20260619 02:45:42.772811 27923 ts_tablet_manager.cc:1397] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:45:42.774776 27925 tablet_bootstrap.cc:492] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Bootstrap starting.
I20260619 02:45:42.774758 27923 raft_consensus.cc:348] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:42.774976 27923 raft_consensus.cc:374] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:42.775022 27923 raft_consensus.cc:729] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb459bd0c9564ce790f7a4af4ef683c2, State: Initialized, Role: FOLLOWER
I20260619 02:45:42.775084 27926 tablet_bootstrap.cc:492] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Bootstrap starting.
I20260619 02:45:42.775138 27923 consensus_queue.cc:260] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:42.775408 27923 ts_tablet_manager.cc:1428] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:45:42.775519 27755 heartbeater.cc:503] Master 127.19.113.126:33279 was elected leader, sending a full tablet report...
I20260619 02:45:42.775740 27926 tablet_bootstrap.cc:654] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:42.775756 27925 tablet_bootstrap.cc:654] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:42.776083 27925 log.cc:824] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:42.776084 27926 log.cc:824] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:42.776810 27926 tablet_bootstrap.cc:492] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: No bootstrap required, opened a new log
I20260619 02:45:42.776867 27926 ts_tablet_manager.cc:1397] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:42.777179 27925 tablet_bootstrap.cc:492] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: No bootstrap required, opened a new log
I20260619 02:45:42.777238 27925 ts_tablet_manager.cc:1397] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:45:42.778242 27926 raft_consensus.cc:348] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:42.778403 27926 raft_consensus.cc:374] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:42.778457 27926 raft_consensus.cc:729] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c77bb1920234a3aad0959cf780b26d9, State: Initialized, Role: FOLLOWER
I20260619 02:45:42.778584 27926 consensus_queue.cc:260] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:42.778829 27926 ts_tablet_manager.cc:1428] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:42.778895 27895 heartbeater.cc:503] Master 127.19.113.126:33279 was elected leader, sending a full tablet report...
I20260619 02:45:42.778800 27925 raft_consensus.cc:348] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:42.778987 27925 raft_consensus.cc:374] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:42.779027 27925 raft_consensus.cc:729] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2191278f82f447f3a8c08f1813fd5c68, State: Initialized, Role: FOLLOWER
I20260619 02:45:42.779134 27925 consensus_queue.cc:260] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:42.779372 27925 ts_tablet_manager.cc:1428] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:45:42.779415 27615 heartbeater.cc:503] Master 127.19.113.126:33279 was elected leader, sending a full tablet report...
W20260619 02:45:42.799768 27756 tablet.cc:2369] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:45:42.901206 27616 tablet.cc:2369] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:45:42.972045 27896 tablet.cc:2369] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:45:43.069831 27930 raft_consensus.cc:482] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:43.069948 27930 raft_consensus.cc:504] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.070281 27930 leader_election.cc:288] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671), cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827)
I20260619 02:45:43.074020 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.070537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:50923 (local address 127.19.113.66:44827)
0619 02:45:43.070578 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.070579 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.070588 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:43.070646 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.070647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.070656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.070705 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:43.070886 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.071653 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.073120 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.073668 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.073740 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.073749 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.073905 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.073939 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:45:43.074015 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.070527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:58623 (local address 127.19.113.65:39671)
0619 02:45:43.070574 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.070576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.070744 (+ 168us) server_negotiation.cc:407] Connection header received
0619 02:45:43.070782 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.070782 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.070788 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.070820 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:43.070964 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.071651 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.073119 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.073663 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.073740 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.073749 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.073905 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.073941 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:43.074033 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.070478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39671 (local address 127.19.113.67:58623)
0619 02:45:43.070681 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.070688 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.070738 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.070836 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.070837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.070838 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:43.070922 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.070923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.071743 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.071745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.073027 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.073029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.073741 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.073741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.073833 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.073838 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.073895 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.073933 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":85,"thread_start_us":96,"threads_started":1}
I20260619 02:45:43.074116 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.070474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:44827 (local address 127.19.113.67:50923)
0619 02:45:43.070525 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.070538 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.070588 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.070731 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.070732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.070734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:43.070802 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.070804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.071743 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.071745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.073020 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.073024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.073741 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.073742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.073836 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.073838 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.073895 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.073933 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":14}
I20260619 02:45:43.074405 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "631ead3d29494490931b2f3b7c30b530" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2191278f82f447f3a8c08f1813fd5c68" is_pre_election: true
I20260619 02:45:43.074474 27709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "631ead3d29494490931b2f3b7c30b530" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" is_pre_election: true
I20260619 02:45:43.074566 27569 raft_consensus.cc:2454] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 0.
I20260619 02:45:43.074617 27709 raft_consensus.cc:2454] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 0.
I20260619 02:45:43.074803 27780 leader_election.cc:302] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2191278f82f447f3a8c08f1813fd5c68, 7c77bb1920234a3aad0959cf780b26d9; no voters:
I20260619 02:45:43.074980 27930 raft_consensus.cc:2781] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:43.075040 27930 raft_consensus.cc:482] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:43.075057 27930 raft_consensus.cc:3037] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.075933 27930 raft_consensus.cc:504] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.076061 27930 leader_election.cc:288] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 election: Requested vote from peers 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671), cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827)
I20260619 02:45:43.076215 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "631ead3d29494490931b2f3b7c30b530" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2191278f82f447f3a8c08f1813fd5c68"
I20260619 02:45:43.076222 27709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "631ead3d29494490931b2f3b7c30b530" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb459bd0c9564ce790f7a4af4ef683c2"
I20260619 02:45:43.076287 27569 raft_consensus.cc:3037] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.076288 27709 raft_consensus.cc:3037] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.076858 27569 raft_consensus.cc:2454] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 1.
I20260619 02:45:43.077006 27709 raft_consensus.cc:2454] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 1.
I20260619 02:45:43.077016 27780 leader_election.cc:302] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2191278f82f447f3a8c08f1813fd5c68, 7c77bb1920234a3aad0959cf780b26d9; no voters:
I20260619 02:45:43.077168 27930 raft_consensus.cc:2781] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:43.077302 27930 raft_consensus.cc:686] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 LEADER]: Becoming Leader. State: Replica: 7c77bb1920234a3aad0959cf780b26d9, State: Running, Role: LEADER
I20260619 02:45:43.077382 27930 consensus_queue.cc:237] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.078205 27412 catalog_manager.cc:5495] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67). New cstate: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260619 02:45:43.120121 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.117005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.19.113.67:41383)
0619 02:45:43.117058 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.117060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.117096 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:45:43.117139 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.117140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.117164 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.117229 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:43.117390 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.118135 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.119228 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.119521 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.119587 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.119755 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.119907 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.119991 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.120015 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.120038 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:45:43.122177 27569 raft_consensus.cc:1261] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Refusing update from remote peer 7c77bb1920234a3aad0959cf780b26d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.122184 27709 raft_consensus.cc:1261] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Refusing update from remote peer 7c77bb1920234a3aad0959cf780b26d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.122488 27933 consensus_queue.cc:1035] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.122620 27930 consensus_queue.cc:1035] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.125125 27937 mvcc.cc:205] Tried to move back new op lower bound from 7298404938225745920 to 7298404938045382656. Current Snapshot: MvccSnapshot[applied={T|T < 7298404938225745920}]
I20260619 02:45:43.125828 27938 mvcc.cc:205] Tried to move back new op lower bound from 7298404938225745920 to 7298404938045382656. Current Snapshot: MvccSnapshot[applied={T|T < 7298404938225745920 or (T in {7298404938225745920})}]
I20260619 02:45:43.128103 27411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:43.128198 27411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:43.129000 27411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: Sending DeleteTablet for 3 replicas of tablet 631ead3d29494490931b2f3b7c30b530
I20260619 02:45:43.129339 27829 tablet_service.cc:1514] Processing DeleteTablet for tablet 631ead3d29494490931b2f3b7c30b530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:45958
I20260619 02:45:43.129329 27689 tablet_service.cc:1514] Processing DeleteTablet for tablet 631ead3d29494490931b2f3b7c30b530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:59928
I20260619 02:45:43.129523 27946 tablet_replica.cc:330] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: stopping tablet replica
I20260619 02:45:43.129634 27946 raft_consensus.cc:2229] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:43.129629 27549 tablet_service.cc:1514] Processing DeleteTablet for tablet 631ead3d29494490931b2f3b7c30b530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:54690
I20260619 02:45:43.129768 27946 raft_consensus.cc:2258] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.130012 27411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:43.130162 27411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:43.130451 27946 ts_tablet_manager.cc:1905] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.129936 27947 tablet_replica.cc:330] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: stopping tablet replica
I20260619 02:45:43.130602 27947 raft_consensus.cc:2229] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:43.130759 27947 raft_consensus.cc:2258] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.131017 27948 tablet_replica.cc:330] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: stopping tablet replica
I20260619 02:45:43.131126 27948 raft_consensus.cc:2229] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:43.131225 27948 raft_consensus.cc:2258] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.131268 27947 ts_tablet_manager.cc:1905] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.131736 27948 ts_tablet_manager.cc:1905] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.131842 27829 tablet_service.cc:1467] Processing CreateTablet for tablet 37dbffde7a354f2da5b2e96ff87d2656 (DEFAULT_TABLE table=test-table [id=6461abb6aec2496abdf01d3080cbdc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.131963 27829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37dbffde7a354f2da5b2e96ff87d2656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.132659 27946 ts_tablet_manager.cc:1918] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:43.132776 27946 log.cc:1196] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/631ead3d29494490931b2f3b7c30b530
I20260619 02:45:43.133023 27946 ts_tablet_manager.cc:1939] T 631ead3d29494490931b2f3b7c30b530 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting consensus metadata
I20260619 02:45:43.133225 27926 tablet_bootstrap.cc:492] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: Bootstrap starting.
I20260619 02:45:43.133687 27948 ts_tablet_manager.cc:1918] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:43.133692 27926 tablet_bootstrap.cc:654] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.133795 27948 log.cc:1196] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/631ead3d29494490931b2f3b7c30b530
I20260619 02:45:43.134025 27948 ts_tablet_manager.cc:1939] T 631ead3d29494490931b2f3b7c30b530 P 2191278f82f447f3a8c08f1813fd5c68: Deleting consensus metadata
I20260619 02:45:43.134231 27689 tablet_service.cc:1467] Processing CreateTablet for tablet 37dbffde7a354f2da5b2e96ff87d2656 (DEFAULT_TABLE table=test-table [id=6461abb6aec2496abdf01d3080cbdc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.134326 27689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37dbffde7a354f2da5b2e96ff87d2656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.134308 27398 catalog_manager.cc:4841] TS cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827): tablet 631ead3d29494490931b2f3b7c30b530 (table test-table [id=ed1a95395fb94cb6aa6ff46cacda6e36]) successfully deleted
I20260619 02:45:43.134377 27926 tablet_bootstrap.cc:492] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: No bootstrap required, opened a new log
I20260619 02:45:43.134410 27926 ts_tablet_manager.cc:1397] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:43.134543 27926 raft_consensus.cc:348] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.134632 27926 raft_consensus.cc:374] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.134651 27926 raft_consensus.cc:729] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c77bb1920234a3aad0959cf780b26d9, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.134692 27926 consensus_queue.cc:260] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.134770 27926 ts_tablet_manager.cc:1428] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:43.134760 27947 ts_tablet_manager.cc:1918] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:43.135274 27947 log.cc:1196] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/631ead3d29494490931b2f3b7c30b530
I20260619 02:45:43.135538 27923 tablet_bootstrap.cc:492] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: Bootstrap starting.
I20260619 02:45:43.135568 27549 tablet_service.cc:1467] Processing CreateTablet for tablet 37dbffde7a354f2da5b2e96ff87d2656 (DEFAULT_TABLE table=test-table [id=6461abb6aec2496abdf01d3080cbdc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.135593 27947 ts_tablet_manager.cc:1939] T 631ead3d29494490931b2f3b7c30b530 P 7c77bb1920234a3aad0959cf780b26d9: Deleting consensus metadata
I20260619 02:45:43.135654 27549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37dbffde7a354f2da5b2e96ff87d2656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.135876 27923 tablet_bootstrap.cc:654] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.135965 27398 catalog_manager.cc:4841] TS 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383): tablet 631ead3d29494490931b2f3b7c30b530 (table test-table [id=ed1a95395fb94cb6aa6ff46cacda6e36]) successfully deleted
I20260619 02:45:43.136759 27923 tablet_bootstrap.cc:492] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: No bootstrap required, opened a new log
I20260619 02:45:43.136811 27923 ts_tablet_manager.cc:1397] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:43.136916 27396 catalog_manager.cc:4841] TS 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671): tablet 631ead3d29494490931b2f3b7c30b530 (table test-table [id=ed1a95395fb94cb6aa6ff46cacda6e36]) successfully deleted
I20260619 02:45:43.136952 27923 raft_consensus.cc:348] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.137027 27923 raft_consensus.cc:374] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.137051 27923 raft_consensus.cc:729] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb459bd0c9564ce790f7a4af4ef683c2, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.137089 27923 consensus_queue.cc:260] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.137445 27923 ts_tablet_manager.cc:1428] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:43.137558 27925 tablet_bootstrap.cc:492] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: Bootstrap starting.
I20260619 02:45:43.137933 27925 tablet_bootstrap.cc:654] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.139045 27925 tablet_bootstrap.cc:492] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: No bootstrap required, opened a new log
I20260619 02:45:43.139088 27925 ts_tablet_manager.cc:1397] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:45:43.139219 27925 raft_consensus.cc:348] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.139273 27925 raft_consensus.cc:374] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.139289 27925 raft_consensus.cc:729] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2191278f82f447f3a8c08f1813fd5c68, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.139328 27925 consensus_queue.cc:260] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.139716 27925 ts_tablet_manager.cc:1428] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:43.231882 27931 raft_consensus.cc:482] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:43.231978 27931 raft_consensus.cc:504] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.232312 27931 leader_election.cc:288] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827), 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383)
I20260619 02:45:43.236099 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.232741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:39033 (local address 127.19.113.67:41383)
0619 02:45:43.232786 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.232788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.232938 (+ 150us) server_negotiation.cc:407] Connection header received
0619 02:45:43.233030 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.233031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.233042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.233066 (+ 24us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:43.233253 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.233703 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.235199 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.235729 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.235807 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.235818 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.235997 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.236038 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:45:43.236111 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.232681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:41383 (local address 127.19.113.65:39033)
0619 02:45:43.232853 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.232862 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.232961 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.233146 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.233147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.233148 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:43.233196 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.233197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.233773 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.233774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.235117 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.235118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.235834 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.235835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.235899 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.235903 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.235962 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.236021 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":18,"thread_start_us":71,"threads_started":1}
I20260619 02:45:43.236749 27952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.232509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:44827 (local address 127.19.113.65:59939)
0619 02:45:43.232632 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.232643 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.232734 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.233019 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.233020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.233022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:43.233100 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.233102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.233923 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.233924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.235114 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.235117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.236464 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.236465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.236545 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.236549 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.236632 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.236674 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260619 02:45:43.236783 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.232560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:59939 (local address 127.19.113.66:44827)
0619 02:45:43.232611 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.232613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.232705 (+ 92us) server_negotiation.cc:407] Connection header received
0619 02:45:43.232925 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.232927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.232940 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.233008 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:43.233168 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.233846 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.235854 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.236381 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.236453 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.236461 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.236667 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.236713 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:45:43.236729 27849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37dbffde7a354f2da5b2e96ff87d2656" candidate_uuid: "2191278f82f447f3a8c08f1813fd5c68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c77bb1920234a3aad0959cf780b26d9" is_pre_election: true
I20260619 02:45:43.236886 27849 raft_consensus.cc:2454] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2191278f82f447f3a8c08f1813fd5c68 in term 0.
I20260619 02:45:43.237035 27709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37dbffde7a354f2da5b2e96ff87d2656" candidate_uuid: "2191278f82f447f3a8c08f1813fd5c68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" is_pre_election: true
I20260619 02:45:43.237113 27709 raft_consensus.cc:2454] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2191278f82f447f3a8c08f1813fd5c68 in term 0.
I20260619 02:45:43.237078 27502 leader_election.cc:302] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2191278f82f447f3a8c08f1813fd5c68, 7c77bb1920234a3aad0959cf780b26d9; no voters:
I20260619 02:45:43.237200 27931 raft_consensus.cc:2781] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:43.237224 27931 raft_consensus.cc:482] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:43.237241 27931 raft_consensus.cc:3037] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.238075 27931 raft_consensus.cc:504] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.238209 27931 leader_election.cc:288] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [CANDIDATE]: Term 1 election: Requested vote from peers cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827), 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383)
I20260619 02:45:43.238373 27849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37dbffde7a354f2da5b2e96ff87d2656" candidate_uuid: "2191278f82f447f3a8c08f1813fd5c68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c77bb1920234a3aad0959cf780b26d9"
I20260619 02:45:43.238401 27709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37dbffde7a354f2da5b2e96ff87d2656" candidate_uuid: "2191278f82f447f3a8c08f1813fd5c68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb459bd0c9564ce790f7a4af4ef683c2"
I20260619 02:45:43.238441 27849 raft_consensus.cc:3037] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.238461 27709 raft_consensus.cc:3037] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.239091 27709 raft_consensus.cc:2454] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2191278f82f447f3a8c08f1813fd5c68 in term 1.
I20260619 02:45:43.239171 27849 raft_consensus.cc:2454] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2191278f82f447f3a8c08f1813fd5c68 in term 1.
I20260619 02:45:43.239238 27502 leader_election.cc:302] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2191278f82f447f3a8c08f1813fd5c68, cb459bd0c9564ce790f7a4af4ef683c2; no voters:
I20260619 02:45:43.239336 27931 raft_consensus.cc:2781] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:43.239496 27931 raft_consensus.cc:686] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 LEADER]: Becoming Leader. State: Replica: 2191278f82f447f3a8c08f1813fd5c68, State: Running, Role: LEADER
I20260619 02:45:43.239578 27931 consensus_queue.cc:237] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.240263 27411 catalog_manager.cc:5495] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65). New cstate: current_term: 1 leader_uuid: "2191278f82f447f3a8c08f1813fd5c68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:45:43 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837142, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.126@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:43.277639 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.271525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39462 (local address 127.19.113.126:33279)
0619 02:45:43.271674 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.271676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.271684 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:43.271704 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.271705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.271714 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.271768 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:43.272200 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.272699 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.273375 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.273580 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.273622 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.275532 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:43.275534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:43.275534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:43.275540 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:43.276756 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:43.277080 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:43.277080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:43.277081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:43.277126 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:43.277256 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:43.277256 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:43.277257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:43.277349 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:43.277391 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.277493 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.277566 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260619 02:45:43.277782 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.271463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:33279 (local address 127.0.0.1:39462)
0619 02:45:43.271570 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.271583 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.271655 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.271779 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.271779 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.272097 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:43.272144 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.272147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.272765 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.272766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.273319 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.273321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.273629 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.273630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.273667 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.274128 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:43.274132 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:43.275460 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:43.276826 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:43.276832 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:43.276834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:43.277026 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:43.277169 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:43.277170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:43.277170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:43.277218 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:43.277389 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:43.277392 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:43.277430 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.277474 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.277642 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":78,"thread_start_us":43,"threads_started":1}
I20260619 02:45:43.279629 27411 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:39462:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:45:43.279819 27411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:43.281240 27549 tablet_service.cc:1467] Processing CreateTablet for tablet ce56c02fb1e04ebbb48c05cfbd2d9ad5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b4339086346436a8c9185f80da962cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:43.281277 27689 tablet_service.cc:1467] Processing CreateTablet for tablet ce56c02fb1e04ebbb48c05cfbd2d9ad5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b4339086346436a8c9185f80da962cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:43.281420 27549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce56c02fb1e04ebbb48c05cfbd2d9ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.281420 27689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce56c02fb1e04ebbb48c05cfbd2d9ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.283011 27925 tablet_bootstrap.cc:492] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: Bootstrap starting.
I20260619 02:45:43.283030 27829 tablet_service.cc:1467] Processing CreateTablet for tablet ce56c02fb1e04ebbb48c05cfbd2d9ad5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b4339086346436a8c9185f80da962cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:45:43.283164 27829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce56c02fb1e04ebbb48c05cfbd2d9ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.283417 27925 tablet_bootstrap.cc:654] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.283761 27923 tablet_bootstrap.cc:492] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: Bootstrap starting.
I20260619 02:45:43.284055 27925 tablet_bootstrap.cc:492] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: No bootstrap required, opened a new log
I20260619 02:45:43.284098 27925 ts_tablet_manager.cc:1397] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:43.284192 27923 tablet_bootstrap.cc:654] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.284235 27925 raft_consensus.cc:348] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.284300 27925 raft_consensus.cc:374] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.284313 27925 raft_consensus.cc:729] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2191278f82f447f3a8c08f1813fd5c68, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.284353 27925 consensus_queue.cc:260] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.284437 27925 ts_tablet_manager.cc:1428] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:43.284543 27931 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.284605 27931 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:43.284705 27926 tablet_bootstrap.cc:492] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: Bootstrap starting.
I20260619 02:45:43.284997 27926 tablet_bootstrap.cc:654] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.285130 27923 tablet_bootstrap.cc:492] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: No bootstrap required, opened a new log
I20260619 02:45:43.285178 27923 ts_tablet_manager.cc:1397] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:45:43.285315 27923 raft_consensus.cc:348] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.285384 27923 raft_consensus.cc:374] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.285408 27923 raft_consensus.cc:729] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb459bd0c9564ce790f7a4af4ef683c2, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.285446 27923 consensus_queue.cc:260] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.285531 27923 ts_tablet_manager.cc:1428] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:43.285545 27926 tablet_bootstrap.cc:492] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: No bootstrap required, opened a new log
I20260619 02:45:43.285586 27926 ts_tablet_manager.cc:1397] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:43.285676 27927 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.285691 27926 raft_consensus.cc:348] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.285739 27926 raft_consensus.cc:374] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.285753 27926 raft_consensus.cc:729] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c77bb1920234a3aad0959cf780b26d9, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.285739 27927 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:43.285785 27926 consensus_queue.cc:260] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.285869 27926 ts_tablet_manager.cc:1428] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:43.285928 27941 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.286017 27941 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:43.334244 27941 raft_consensus.cc:482] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:43.334368 27941 raft_consensus.cc:504] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.334532 27941 leader_election.cc:288] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671), cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827)
I20260619 02:45:43.334750 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce56c02fb1e04ebbb48c05cfbd2d9ad5" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2191278f82f447f3a8c08f1813fd5c68" is_pre_election: true
I20260619 02:45:43.334734 27709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce56c02fb1e04ebbb48c05cfbd2d9ad5" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" is_pre_election: true
I20260619 02:45:43.334859 27569 raft_consensus.cc:2454] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 0.
I20260619 02:45:43.334856 27709 raft_consensus.cc:2454] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 0.
I20260619 02:45:43.335041 27780 leader_election.cc:302] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2191278f82f447f3a8c08f1813fd5c68, 7c77bb1920234a3aad0959cf780b26d9; no voters:
I20260619 02:45:43.335141 27941 raft_consensus.cc:2781] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:43.335184 27941 raft_consensus.cc:482] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:43.335210 27941 raft_consensus.cc:3037] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.335937 27941 raft_consensus.cc:504] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.336064 27941 leader_election.cc:288] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 election: Requested vote from peers 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671), cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827)
I20260619 02:45:43.336215 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce56c02fb1e04ebbb48c05cfbd2d9ad5" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2191278f82f447f3a8c08f1813fd5c68"
I20260619 02:45:43.336282 27569 raft_consensus.cc:3037] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.336251 27709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce56c02fb1e04ebbb48c05cfbd2d9ad5" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb459bd0c9564ce790f7a4af4ef683c2"
I20260619 02:45:43.336326 27709 raft_consensus.cc:3037] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.336861 27569 raft_consensus.cc:2454] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 1.
I20260619 02:45:43.336920 27709 raft_consensus.cc:2454] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 1.
I20260619 02:45:43.337024 27780 leader_election.cc:302] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2191278f82f447f3a8c08f1813fd5c68, 7c77bb1920234a3aad0959cf780b26d9; no voters:
I20260619 02:45:43.337103 27941 raft_consensus.cc:2781] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:43.337160 27941 raft_consensus.cc:686] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 LEADER]: Becoming Leader. State: Replica: 7c77bb1920234a3aad0959cf780b26d9, State: Running, Role: LEADER
I20260619 02:45:43.337218 27941 consensus_queue.cc:237] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.337466 27930 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: TxnStatusTablet state changed. Reason: New leader 7c77bb1920234a3aad0959cf780b26d9. Latest consensus state: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.337541 27930 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:43.337700 27967 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:45:43.337821 27967 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:45:43.337831 27412 catalog_manager.cc:5495] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67). New cstate: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260619 02:45:43.344950 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.342421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45968 (local address 127.19.113.67:41383)
0619 02:45:43.342480 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.342482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.342540 (+ 58us) server_negotiation.cc:407] Connection header received
0619 02:45:43.342624 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.342625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.342633 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.342675 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:43.342791 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.343289 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.344350 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.344539 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.344582 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.344749 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.344774 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.344804 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.344880 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.344898 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:43.345010 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.342362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:41383 (local address 127.0.0.1:45968)
0619 02:45:43.342489 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.342501 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.342558 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.342685 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.342685 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.342687 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:43.342738 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.342740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.343354 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.343356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.344262 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.344265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.344597 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.344598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.344661 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.344678 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.344825 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.344830 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.344879 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.344930 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 02:45:43.345888 27569 raft_consensus.cc:1261] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Refusing update from remote peer 7c77bb1920234a3aad0959cf780b26d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.346102 27709 raft_consensus.cc:1261] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Refusing update from remote peer 7c77bb1920234a3aad0959cf780b26d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.346197 27941 consensus_queue.cc:1035] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.346295 27930 consensus_queue.cc:1035] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.346725 27931 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: TxnStatusTablet state changed. Reason: New leader 7c77bb1920234a3aad0959cf780b26d9. Latest consensus state: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.346822 27931 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:43.346971 27927 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: TxnStatusTablet state changed. Reason: New leader 7c77bb1920234a3aad0959cf780b26d9. Latest consensus state: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.347055 27927 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:43.347422 27941 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.347508 27941 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:43.347821 27936 mvcc.cc:205] Tried to move back new op lower bound from 7298404939143299072 to 7298404939109666816. Current Snapshot: MvccSnapshot[applied={T|T < 7298404939143299072 or (T in {7298404939143299072})}]
I20260619 02:45:43.348515 27931 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.348598 27931 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 2191278f82f447f3a8c08f1813fd5c68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:43.351539 27941 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.351611 27941 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P 7c77bb1920234a3aad0959cf780b26d9: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:45:43.351941 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.348932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.19.113.65:39671)
0619 02:45:43.349074 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.349075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.349082 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:43.349113 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.349114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.349133 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.349189 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:43.349316 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.350088 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.350819 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.351151 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.351215 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.351364 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.351493 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.351578 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.351832 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.351853 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260619 02:45:43.352072 27927 tablet_replica.cc:439] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } } }
I20260619 02:45:43.352152 27927 tablet_replica.cc:442] T ce56c02fb1e04ebbb48c05cfbd2d9ad5 P cb459bd0c9564ce790f7a4af4ef683c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:45:43.356626 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.354346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:37251 (local address 127.19.113.67:41383)
0619 02:45:43.354386 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.354388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.354510 (+ 122us) server_negotiation.cc:407] Connection header received
0619 02:45:43.354567 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.354568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.354579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.354619 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:43.354755 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.355263 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.356024 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.356213 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.356263 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.356412 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.356445 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.356485 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.356534 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.356554 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:45:43.356663 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.354305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:41383 (local address 127.19.113.65:37251)
0619 02:45:43.354445 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.354462 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.354512 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.354635 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.354636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.354637 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:43.354700 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.354701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.355324 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.355325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.355946 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.355948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.356267 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.356268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.356330 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.356346 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.356490 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.356492 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.356532 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.356575 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260619 02:45:43.359036 27849 raft_consensus.cc:1261] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Refusing update from remote peer 2191278f82f447f3a8c08f1813fd5c68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.359187 27709 raft_consensus.cc:1261] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Refusing update from remote peer 2191278f82f447f3a8c08f1813fd5c68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.359306 27954 consensus_queue.cc:1035] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.359475 27954 consensus_queue.cc:1035] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.370020 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.367228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39671 (local address 127.19.113.67:45595)
0619 02:45:43.367398 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.367408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.367475 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.367589 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.367590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.367592 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:43.367643 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.367645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.368497 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.368498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.369244 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.369247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.369686 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.369687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.369727 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.369742 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.369892 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.369893 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.369922 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.369956 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260619 02:45:43.370028 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.367283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:45595 (local address 127.19.113.65:39671)
0619 02:45:43.367393 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.367395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.367448 (+ 53us) server_negotiation.cc:407] Connection header received
0619 02:45:43.367524 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.367525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.367535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.367577 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:43.367695 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.368431 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.369317 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.369619 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.369682 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.369811 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.369844 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.369890 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.369928 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.369949 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 02:45:43.380117 27412 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:43.380232 27412 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:43.380818 27412 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: Sending DeleteTablet for 3 replicas of tablet 37dbffde7a354f2da5b2e96ff87d2656
I20260619 02:45:43.381068 27689 tablet_service.cc:1514] Processing DeleteTablet for tablet 37dbffde7a354f2da5b2e96ff87d2656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:59928
I20260619 02:45:43.381136 27549 tablet_service.cc:1514] Processing DeleteTablet for tablet 37dbffde7a354f2da5b2e96ff87d2656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:54690
I20260619 02:45:43.381163 27946 tablet_replica.cc:330] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: stopping tablet replica
I20260619 02:45:43.381206 27946 raft_consensus.cc:2229] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:43.381219 27948 tablet_replica.cc:330] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: stopping tablet replica
I20260619 02:45:43.381240 27946 raft_consensus.cc:2258] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.381258 27948 raft_consensus.cc:2229] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:43.381343 27948 raft_consensus.cc:2258] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.381842 27946 ts_tablet_manager.cc:1905] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.383495 27946 ts_tablet_manager.cc:1918] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:43.383558 27946 log.cc:1196] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/37dbffde7a354f2da5b2e96ff87d2656
I20260619 02:45:43.383769 27946 ts_tablet_manager.cc:1939] T 37dbffde7a354f2da5b2e96ff87d2656 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting consensus metadata
I20260619 02:45:43.384085 27398 catalog_manager.cc:4841] TS cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827): tablet 37dbffde7a354f2da5b2e96ff87d2656 (table test-table [id=6461abb6aec2496abdf01d3080cbdc25]) successfully deleted
I20260619 02:45:43.384109 27948 ts_tablet_manager.cc:1905] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.385617 27948 ts_tablet_manager.cc:1918] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:43.385677 27948 log.cc:1196] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/37dbffde7a354f2da5b2e96ff87d2656
I20260619 02:45:43.385876 27948 ts_tablet_manager.cc:1939] T 37dbffde7a354f2da5b2e96ff87d2656 P 2191278f82f447f3a8c08f1813fd5c68: Deleting consensus metadata
I20260619 02:45:43.386186 27396 catalog_manager.cc:4841] TS 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671): tablet 37dbffde7a354f2da5b2e96ff87d2656 (table test-table [id=6461abb6aec2496abdf01d3080cbdc25]) successfully deleted
I20260619 02:45:43.386266 27829 tablet_service.cc:1514] Processing DeleteTablet for tablet 37dbffde7a354f2da5b2e96ff87d2656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:45958
I20260619 02:45:43.386381 27947 tablet_replica.cc:330] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: stopping tablet replica
I20260619 02:45:43.386439 27947 raft_consensus.cc:2229] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:43.386478 27947 raft_consensus.cc:2258] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.386863 27947 ts_tablet_manager.cc:1905] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.388559 27947 ts_tablet_manager.cc:1918] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:43.388609 27947 log.cc:1196] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/37dbffde7a354f2da5b2e96ff87d2656
I20260619 02:45:43.388795 27947 ts_tablet_manager.cc:1939] T 37dbffde7a354f2da5b2e96ff87d2656 P 7c77bb1920234a3aad0959cf780b26d9: Deleting consensus metadata
I20260619 02:45:43.389278 27398 catalog_manager.cc:4841] TS 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383): tablet 37dbffde7a354f2da5b2e96ff87d2656 (table test-table [id=6461abb6aec2496abdf01d3080cbdc25]) successfully deleted
Jun 19 02:45:43 dist-test-slave-zqpb krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:45:43 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:43 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:45:43.421922 27962 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:43.415042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39466 (local address 127.19.113.126:33279)
0619 02:45:43.415091 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.415093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.415293 (+ 200us) server_negotiation.cc:407] Connection header received
0619 02:45:43.415374 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.415375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.415385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.415452 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:43.416203 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.416944 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.418033 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.418352 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.418417 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.421850 (+ 3433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39466: BlockingRecv error: recv got EOF from 127.0.0.1:39466 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 02:45:43 dist-test-slave-zqpb krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM
I20260619 02:45:43.431757 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.422761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39468 (local address 127.19.113.126:33279)
0619 02:45:43.422798 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.422800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.423052 (+ 252us) server_negotiation.cc:407] Connection header received
0619 02:45:43.423104 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.423105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.423116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.423173 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:43.423726 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.424486 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.425503 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.425806 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.425891 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.428751 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:43.428753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:43.428753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:43.428762 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:43.430411 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:43.430900 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:43.430902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:43.430902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:43.430972 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:43.431199 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:43.431200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:43.431200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:43.431343 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:43.431444 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.431569 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.431648 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:45:43.433475 27412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:43.433655 27412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:43.435045 27689 tablet_service.cc:1467] Processing CreateTablet for tablet d987d3c1ddc241979cf652c961cd8d25 (DEFAULT_TABLE table=test-table [id=7068b3616f5748dc87072aabda2d12e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.435050 27829 tablet_service.cc:1467] Processing CreateTablet for tablet d987d3c1ddc241979cf652c961cd8d25 (DEFAULT_TABLE table=test-table [id=7068b3616f5748dc87072aabda2d12e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.435178 27689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d987d3c1ddc241979cf652c961cd8d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.435178 27829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d987d3c1ddc241979cf652c961cd8d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.436419 27923 tablet_bootstrap.cc:492] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: Bootstrap starting.
I20260619 02:45:43.436522 27926 tablet_bootstrap.cc:492] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: Bootstrap starting.
I20260619 02:45:43.436811 27923 tablet_bootstrap.cc:654] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.436851 27926 tablet_bootstrap.cc:654] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.437613 27926 tablet_bootstrap.cc:492] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: No bootstrap required, opened a new log
I20260619 02:45:43.437661 27926 ts_tablet_manager.cc:1397] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:43.437750 27549 tablet_service.cc:1467] Processing CreateTablet for tablet d987d3c1ddc241979cf652c961cd8d25 (DEFAULT_TABLE table=test-table [id=7068b3616f5748dc87072aabda2d12e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.437811 27926 raft_consensus.cc:348] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.437860 27549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d987d3c1ddc241979cf652c961cd8d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.437876 27926 raft_consensus.cc:374] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.437896 27926 raft_consensus.cc:729] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c77bb1920234a3aad0959cf780b26d9, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.437938 27926 consensus_queue.cc:260] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.438026 27926 ts_tablet_manager.cc:1428] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:43.439188 27925 tablet_bootstrap.cc:492] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: Bootstrap starting.
I20260619 02:45:43.439756 27925 tablet_bootstrap.cc:654] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.439829 27923 tablet_bootstrap.cc:492] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: No bootstrap required, opened a new log
I20260619 02:45:43.439872 27923 ts_tablet_manager.cc:1397] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260619 02:45:43.440052 27923 raft_consensus.cc:348] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.440114 27923 raft_consensus.cc:374] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.440132 27923 raft_consensus.cc:729] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb459bd0c9564ce790f7a4af4ef683c2, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.440178 27923 consensus_queue.cc:260] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.440260 27923 ts_tablet_manager.cc:1428] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:43.443331 27925 tablet_bootstrap.cc:492] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: No bootstrap required, opened a new log
I20260619 02:45:43.443387 27925 ts_tablet_manager.cc:1397] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 02:45:43.443513 27925 raft_consensus.cc:348] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.443580 27925 raft_consensus.cc:374] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.443598 27925 raft_consensus.cc:729] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2191278f82f447f3a8c08f1813fd5c68, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.443640 27925 consensus_queue.cc:260] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.443717 27925 ts_tablet_manager.cc:1428] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:43.636788 27927 raft_consensus.cc:482] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:43.636914 27927 raft_consensus.cc:504] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.637226 27927 leader_election.cc:288] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671), 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383)
I20260619 02:45:43.640179 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.637568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:57633 (local address 127.19.113.67:41383)
0619 02:45:43.637620 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.637622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.637631 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:43.637685 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.637686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.637701 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.637749 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:43.637894 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.638382 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.639502 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.639862 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.639933 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.639943 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.640068 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.640106 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:45:43.640180 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.637477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:43531 (local address 127.19.113.65:39671)
0619 02:45:43.637553 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.637555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.637693 (+ 138us) server_negotiation.cc:407] Connection header received
0619 02:45:43.637736 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.637737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.637752 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.637814 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:45:43.637955 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.638407 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.639499 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.639852 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.639910 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.639920 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.640078 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.640112 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:45:43.640204 27996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.637417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:41383 (local address 127.19.113.66:57633)
0619 02:45:43.637555 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.637567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.637627 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.637758 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.637759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.637761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:43.637823 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.637826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.638475 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.638476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.639408 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.639410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.639960 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.639962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.640006 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.640010 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.640053 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.640097 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":33,"thread_start_us":72,"threads_started":1}
I20260619 02:45:43.640215 27997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.637417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39671 (local address 127.19.113.66:43531)
0619 02:45:43.637610 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.637620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.637684 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.637827 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.637827 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.637829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:45:43.637891 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.637892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.638484 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.638484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.639396 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.639399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.639953 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.639955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.640013 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.640016 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.640063 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.640101 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":36,"thread_start_us":93,"threads_started":1}
I20260619 02:45:43.640494 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d987d3c1ddc241979cf652c961cd8d25" candidate_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2191278f82f447f3a8c08f1813fd5c68" is_pre_election: true
I20260619 02:45:43.640507 27849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d987d3c1ddc241979cf652c961cd8d25" candidate_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c77bb1920234a3aad0959cf780b26d9" is_pre_election: true
I20260619 02:45:43.640579 27569 raft_consensus.cc:2454] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb459bd0c9564ce790f7a4af4ef683c2 in term 0.
I20260619 02:45:43.640588 27849 raft_consensus.cc:2454] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb459bd0c9564ce790f7a4af4ef683c2 in term 0.
I20260619 02:45:43.640758 27640 leader_election.cc:302] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2191278f82f447f3a8c08f1813fd5c68, cb459bd0c9564ce790f7a4af4ef683c2; no voters:
I20260619 02:45:43.640885 27927 raft_consensus.cc:2781] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:43.640935 27927 raft_consensus.cc:482] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:43.640957 27927 raft_consensus.cc:3037] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.641592 27927 raft_consensus.cc:504] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.641703 27927 leader_election.cc:288] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [CANDIDATE]: Term 1 election: Requested vote from peers 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671), 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383)
I20260619 02:45:43.641837 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d987d3c1ddc241979cf652c961cd8d25" candidate_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2191278f82f447f3a8c08f1813fd5c68"
I20260619 02:45:43.641911 27569 raft_consensus.cc:3037] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.641877 27849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d987d3c1ddc241979cf652c961cd8d25" candidate_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c77bb1920234a3aad0959cf780b26d9"
I20260619 02:45:43.642005 27849 raft_consensus.cc:3037] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.642475 27569 raft_consensus.cc:2454] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb459bd0c9564ce790f7a4af4ef683c2 in term 1.
I20260619 02:45:43.642628 27640 leader_election.cc:302] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2191278f82f447f3a8c08f1813fd5c68, cb459bd0c9564ce790f7a4af4ef683c2; no voters:
I20260619 02:45:43.642644 27849 raft_consensus.cc:2454] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb459bd0c9564ce790f7a4af4ef683c2 in term 1.
I20260619 02:45:43.642726 27927 raft_consensus.cc:2781] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:43.642902 27927 raft_consensus.cc:686] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 LEADER]: Becoming Leader. State: Replica: cb459bd0c9564ce790f7a4af4ef683c2, State: Running, Role: LEADER
I20260619 02:45:43.642977 27927 consensus_queue.cc:237] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.643668 27411 catalog_manager.cc:5495] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66). New cstate: current_term: 1 leader_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260619 02:45:43.668494 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.666206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59934 (local address 127.19.113.66:44827)
0619 02:45:43.666258 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.666259 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.666267 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:43.666327 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.666328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.666351 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.666406 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:43.666547 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.667072 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.667804 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.668012 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.668066 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.668230 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.668331 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.668376 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.668414 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.668433 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:45:43.669456 27569 raft_consensus.cc:1261] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Refusing update from remote peer cb459bd0c9564ce790f7a4af4ef683c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.669474 27849 raft_consensus.cc:1261] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Refusing update from remote peer cb459bd0c9564ce790f7a4af4ef683c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.669777 27998 consensus_queue.cc:1035] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.669909 27927 consensus_queue.cc:1035] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.673098 27411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:43.673183 27411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:43.673722 27411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: Sending DeleteTablet for 3 replicas of tablet d987d3c1ddc241979cf652c961cd8d25
I20260619 02:45:43.673947 27689 tablet_service.cc:1514] Processing DeleteTablet for tablet d987d3c1ddc241979cf652c961cd8d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:59928
I20260619 02:45:43.673974 27829 tablet_service.cc:1514] Processing DeleteTablet for tablet d987d3c1ddc241979cf652c961cd8d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:45958
I20260619 02:45:43.674060 27947 tablet_replica.cc:330] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: stopping tablet replica
I20260619 02:45:43.674044 27549 tablet_service.cc:1514] Processing DeleteTablet for tablet d987d3c1ddc241979cf652c961cd8d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:54690
I20260619 02:45:43.674104 27947 raft_consensus.cc:2229] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:43.674121 27946 tablet_replica.cc:330] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: stopping tablet replica
I20260619 02:45:43.674147 27947 raft_consensus.cc:2258] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.674170 27946 raft_consensus.cc:2229] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:43.674237 27946 raft_consensus.cc:2258] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.674296 27948 tablet_replica.cc:330] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: stopping tablet replica
I20260619 02:45:43.674350 27948 raft_consensus.cc:2229] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:43.674398 27948 raft_consensus.cc:2258] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.674523 27947 ts_tablet_manager.cc:1905] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.674566 27411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:45:43.674686 27411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:45:43.674930 27948 ts_tablet_manager.cc:1905] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.675765 27549 tablet_service.cc:1467] Processing CreateTablet for tablet 129581ac057748848e33d3b760fe4e4e (DEFAULT_TABLE table=test-table [id=b1012c8309964107bf2d94c9121f8250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.675851 27549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129581ac057748848e33d3b760fe4e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.676044 27947 ts_tablet_manager.cc:1918] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:43.676095 27947 log.cc:1196] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/d987d3c1ddc241979cf652c961cd8d25
I20260619 02:45:43.676314 27947 ts_tablet_manager.cc:1939] T d987d3c1ddc241979cf652c961cd8d25 P 7c77bb1920234a3aad0959cf780b26d9: Deleting consensus metadata
I20260619 02:45:43.676357 27948 ts_tablet_manager.cc:1918] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:43.676400 27948 log.cc:1196] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/d987d3c1ddc241979cf652c961cd8d25
I20260619 02:45:43.676576 27948 ts_tablet_manager.cc:1939] T d987d3c1ddc241979cf652c961cd8d25 P 2191278f82f447f3a8c08f1813fd5c68: Deleting consensus metadata
I20260619 02:45:43.676826 27946 ts_tablet_manager.cc:1905] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.676915 27925 tablet_bootstrap.cc:492] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68: Bootstrap starting.
I20260619 02:45:43.677208 27925 tablet_bootstrap.cc:654] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.677229 27398 catalog_manager.cc:4841] TS 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383): tablet d987d3c1ddc241979cf652c961cd8d25 (table test-table [id=7068b3616f5748dc87072aabda2d12e1]) successfully deleted
I20260619 02:45:43.677376 27689 tablet_service.cc:1467] Processing CreateTablet for tablet 129581ac057748848e33d3b760fe4e4e (DEFAULT_TABLE table=test-table [id=b1012c8309964107bf2d94c9121f8250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.677479 27689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129581ac057748848e33d3b760fe4e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.678139 27946 ts_tablet_manager.cc:1918] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:45:43.678194 27946 log.cc:1196] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/d987d3c1ddc241979cf652c961cd8d25
I20260619 02:45:43.678366 27946 ts_tablet_manager.cc:1939] T d987d3c1ddc241979cf652c961cd8d25 P cb459bd0c9564ce790f7a4af4ef683c2: Deleting consensus metadata
I20260619 02:45:43.678615 27923 tablet_bootstrap.cc:492] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: Bootstrap starting.
I20260619 02:45:43.678678 27829 tablet_service.cc:1467] Processing CreateTablet for tablet 129581ac057748848e33d3b760fe4e4e (DEFAULT_TABLE table=test-table [id=b1012c8309964107bf2d94c9121f8250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:45:43.678774 27829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129581ac057748848e33d3b760fe4e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:43.678756 27398 catalog_manager.cc:4841] TS cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827): tablet d987d3c1ddc241979cf652c961cd8d25 (table test-table [id=7068b3616f5748dc87072aabda2d12e1]) successfully deleted
I20260619 02:45:43.678979 27923 tablet_bootstrap.cc:654] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.679473 27396 catalog_manager.cc:4841] TS 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671): tablet d987d3c1ddc241979cf652c961cd8d25 (table test-table [id=7068b3616f5748dc87072aabda2d12e1]) successfully deleted
I20260619 02:45:43.679702 27923 tablet_bootstrap.cc:492] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: No bootstrap required, opened a new log
I20260619 02:45:43.679751 27923 ts_tablet_manager.cc:1397] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:43.679880 27923 raft_consensus.cc:348] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.679962 27923 raft_consensus.cc:374] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.679988 27923 raft_consensus.cc:729] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb459bd0c9564ce790f7a4af4ef683c2, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.680006 27926 tablet_bootstrap.cc:492] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: Bootstrap starting.
I20260619 02:45:43.680027 27923 consensus_queue.cc:260] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.680114 27923 ts_tablet_manager.cc:1428] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:43.680323 27926 tablet_bootstrap.cc:654] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:43.680950 27926 tablet_bootstrap.cc:492] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: No bootstrap required, opened a new log
I20260619 02:45:43.680998 27926 ts_tablet_manager.cc:1397] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:43.681104 27926 raft_consensus.cc:348] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.681177 27926 raft_consensus.cc:374] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.681203 27926 raft_consensus.cc:729] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c77bb1920234a3aad0959cf780b26d9, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.681248 27926 consensus_queue.cc:260] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.681336 27926 ts_tablet_manager.cc:1428] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:43.681732 27925 tablet_bootstrap.cc:492] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68: No bootstrap required, opened a new log
I20260619 02:45:43.681802 27925 ts_tablet_manager.cc:1397] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 02:45:43.681916 27925 raft_consensus.cc:348] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.681981 27925 raft_consensus.cc:374] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:43.682006 27925 raft_consensus.cc:729] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2191278f82f447f3a8c08f1813fd5c68, State: Initialized, Role: FOLLOWER
I20260619 02:45:43.682042 27925 consensus_queue.cc:260] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.682132 27925 ts_tablet_manager.cc:1428] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:43.750619 27933 raft_consensus.cc:482] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:45:43.750736 27933 raft_consensus.cc:504] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.750905 27933 leader_election.cc:288] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671), cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827)
I20260619 02:45:43.751082 27709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129581ac057748848e33d3b760fe4e4e" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" is_pre_election: true
I20260619 02:45:43.751080 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129581ac057748848e33d3b760fe4e4e" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2191278f82f447f3a8c08f1813fd5c68" is_pre_election: true
I20260619 02:45:43.751165 27709 raft_consensus.cc:2454] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 0.
I20260619 02:45:43.751194 27569 raft_consensus.cc:2454] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 0.
I20260619 02:45:43.751302 27782 leader_election.cc:302] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c77bb1920234a3aad0959cf780b26d9, cb459bd0c9564ce790f7a4af4ef683c2; no voters:
I20260619 02:45:43.751386 27933 raft_consensus.cc:2781] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:45:43.751422 27933 raft_consensus.cc:482] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:45:43.751441 27933 raft_consensus.cc:3037] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.752132 27933 raft_consensus.cc:504] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.752244 27933 leader_election.cc:288] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 election: Requested vote from peers 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671), cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827)
I20260619 02:45:43.752389 27709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129581ac057748848e33d3b760fe4e4e" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb459bd0c9564ce790f7a4af4ef683c2"
I20260619 02:45:43.752398 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129581ac057748848e33d3b760fe4e4e" candidate_uuid: "7c77bb1920234a3aad0959cf780b26d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2191278f82f447f3a8c08f1813fd5c68"
I20260619 02:45:43.752465 27709 raft_consensus.cc:3037] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.752496 27569 raft_consensus.cc:3037] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:43.753151 27569 raft_consensus.cc:2454] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 1.
I20260619 02:45:43.753151 27709 raft_consensus.cc:2454] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c77bb1920234a3aad0959cf780b26d9 in term 1.
I20260619 02:45:43.753324 27782 leader_election.cc:302] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c77bb1920234a3aad0959cf780b26d9, cb459bd0c9564ce790f7a4af4ef683c2; no voters:
I20260619 02:45:43.753400 27933 raft_consensus.cc:2781] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:43.753438 27933 raft_consensus.cc:686] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 1 LEADER]: Becoming Leader. State: Replica: 7c77bb1920234a3aad0959cf780b26d9, State: Running, Role: LEADER
I20260619 02:45:43.753484 27933 consensus_queue.cc:237] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } }
I20260619 02:45:43.754087 27411 catalog_manager.cc:5495] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67). New cstate: current_term: 1 leader_uuid: "7c77bb1920234a3aad0959cf780b26d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c77bb1920234a3aad0959cf780b26d9" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260619 02:45:43.778327 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.775893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.19.113.67:41383)
0619 02:45:43.775931 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.775933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.775941 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:43.775985 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.775986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.775996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.776055 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:43.776150 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.776874 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.777581 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.777837 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.777921 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.778021 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.778054 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.778105 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.778219 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.778244 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:45:43.782831 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.780464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:55163 (local address 127.19.113.67:41383)
0619 02:45:43.780502 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:43.780504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:43.780512 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:43.780531 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:43.780532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:43.780541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:43.780593 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:43.780685 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.781301 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.782118 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.782410 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.782471 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.782557 (+ 86us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.782586 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.782626 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:43.782726 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:45:43.782746 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:43.782980 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.780350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:41383 (local address 127.19.113.67:55163)
0619 02:45:43.780400 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.780410 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:43.780446 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:43.780595 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:43.780596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:43.780597 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:45:43.780636 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.780637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.781375 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.781376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.782053 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:43.782055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:43.782478 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:43.782478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:43.782514 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:43.782519 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:45:43.782667 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:45:43.782670 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:45:43.782715 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 02:45:43.782757 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260619 02:45:43.785025 27569 raft_consensus.cc:1261] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Refusing update from remote peer 7c77bb1920234a3aad0959cf780b26d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.785074 27709 raft_consensus.cc:1261] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Refusing update from remote peer 7c77bb1920234a3aad0959cf780b26d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:43.785238 27933 consensus_queue.cc:1035] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2191278f82f447f3a8c08f1813fd5c68" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.785351 27930 consensus_queue.cc:1035] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb459bd0c9564ce790f7a4af4ef683c2" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:43.795570 27412 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:43.795668 27412 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:45:43.796303 27412 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8b4353607844288b56c1c31128f79b2: Sending DeleteTablet for 3 replicas of tablet 129581ac057748848e33d3b760fe4e4e
I20260619 02:45:43.796556 27549 tablet_service.cc:1514] Processing DeleteTablet for tablet 129581ac057748848e33d3b760fe4e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:54690
I20260619 02:45:43.796559 27689 tablet_service.cc:1514] Processing DeleteTablet for tablet 129581ac057748848e33d3b760fe4e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:59928
I20260619 02:45:43.796676 27948 tablet_replica.cc:330] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68: stopping tablet replica
I20260619 02:45:43.796741 27829 tablet_service.cc:1514] Processing DeleteTablet for tablet 129581ac057748848e33d3b760fe4e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:45:43 UTC) from {username='oryx', principal='oryx/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:45958
I20260619 02:45:43.796731 27948 raft_consensus.cc:2229] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:43.796830 27948 raft_consensus.cc:2258] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.796845 27947 tablet_replica.cc:330] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: stopping tablet replica
I20260619 02:45:43.796908 27947 raft_consensus.cc:2229] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:45:43.796909 27946 tablet_replica.cc:330] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: stopping tablet replica
I20260619 02:45:43.796962 27946 raft_consensus.cc:2229] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:45:43.796974 27947 raft_consensus.cc:2258] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.797012 27946 raft_consensus.cc:2258] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:45:43.797236 27947 ts_tablet_manager.cc:1905] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.797356 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 27484
I20260619 02:45:43.797396 27948 ts_tablet_manager.cc:1905] T 129581ac057748848e33d3b760fe4e4e P 2191278f82f447f3a8c08f1813fd5c68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.798033 27946 ts_tablet_manager.cc:1905] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:45:43.798794 27947 ts_tablet_manager.cc:1918] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:43.798851 27947 log.cc:1196] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/129581ac057748848e33d3b760fe4e4e
I20260619 02:45:43.799049 27947 ts_tablet_manager.cc:1939] T 129581ac057748848e33d3b760fe4e4e P 7c77bb1920234a3aad0959cf780b26d9: Deleting consensus metadata
I20260619 02:45:43.799437 27946 ts_tablet_manager.cc:1918] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:45:43.799490 27946 log.cc:1196] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/129581ac057748848e33d3b760fe4e4e
I20260619 02:45:43.799490 27398 catalog_manager.cc:4841] TS 7c77bb1920234a3aad0959cf780b26d9 (127.19.113.67:41383): tablet 129581ac057748848e33d3b760fe4e4e (table test-table [id=b1012c8309964107bf2d94c9121f8250]) successfully deleted
I20260619 02:45:43.799674 27946 ts_tablet_manager.cc:1939] T 129581ac057748848e33d3b760fe4e4e P cb459bd0c9564ce790f7a4af4ef683c2: Deleting consensus metadata
I20260619 02:45:43.800026 27398 catalog_manager.cc:4841] TS cb459bd0c9564ce790f7a4af4ef683c2 (127.19.113.66:44827): tablet 129581ac057748848e33d3b760fe4e4e (table test-table [id=b1012c8309964107bf2d94c9121f8250]) successfully deleted
W20260619 02:45:43.803951 27396 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:39671 (error 108)
I20260619 02:45:43.804216 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 27624
I20260619 02:45:43.805231 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:43.804364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39671 (local address 127.0.0.1:54712)
0619 02:45:43.805137 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:43.805189 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:39671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":58,"threads_started":1}
W20260619 02:45:43.805361 27396 catalog_manager.cc:4584] TS 2191278f82f447f3a8c08f1813fd5c68 (127.19.113.65:39671): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 129581ac057748848e33d3b760fe4e4e: Network error: Client connection negotiation failed: client connection to 127.19.113.65:39671: connect: Connection refused (error 111)
I20260619 02:45:43.809906 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 27764
I20260619 02:45:43.814754 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 27380
2026-06-19T02:45:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2782 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:43 dist-test-slave-zqpb krb5kdc[28015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:43 dist-test-slave-zqpb krb5kdc[28015](info): set up 2 sockets
Jun 19 02:45:43 dist-test-slave-zqpb krb5kdc[28015](info): commencing operation
krb5kdc: starting...
W20260619 02:45:44.771165 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:44 dist-test-slave-zqpb krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:44.885766 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:37727
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:37727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:44.965927 28028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:44.966169 28028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:44.966202 28028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:44.967474 28028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:44.967521 28028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:44.967540 28028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:44.967555 28028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:44.967571 28028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:44.969074 28028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:37727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:37727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:44.969455 28028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:44.969702 28028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:44.974397 28036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:44.974522 28033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:44.974651 28028 server_base.cc:1034] running on GCE node
W20260619 02:45:44.974725 28034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:44.975046 28028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:44.975346 28028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:44.976497 28028 hybrid_clock.cc:648] HybridClock initialized: now 1781837144976481 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T02:45:44Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1169 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:45 dist-test-slave-zqpb krb5kdc[28041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:45 dist-test-slave-zqpb krb5kdc[28041](info): set up 2 sockets
Jun 19 02:45:45 dist-test-slave-zqpb krb5kdc[28041](info): commencing operation
krb5kdc: starting...
W20260619 02:45:45.947154 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:45:46.057192 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36125
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:36125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:46.136624 28054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:46.136849 28054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:46.136881 28054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:46.138057 28054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:46.138100 28054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:46.138119 28054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:46.138135 28054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:46.138150 28054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:46.139674 28054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:36125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:46.139930 28054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:46.140144 28054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:46.144759 28062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:46.144766 28059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:46.144824 28054 server_base.cc:1034] running on GCE node
W20260619 02:45:46.144799 28060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:46.145279 28054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:46.145603 28054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:46.146745 28054 hybrid_clock.cc:648] HybridClock initialized: now 1781837146146731 us; error 29 us; skew 500 ppm
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:46.154398 28054 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:46.154834 28054 webserver.cc:460] Webserver started at http://127.19.113.126:40769/ using document root <none> and password file <none>
I20260619 02:45:46.155072 28054 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:46.155119 28054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:46.155218 28054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:46.156220 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "6af69bc7c2fa4962976d266d39f0850b"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "a30cde0875fd38ee46d50527742990a2"
server_key_iv: "c47b9f6ddb07e045018dc3f13ed05e73"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.156527 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "6af69bc7c2fa4962976d266d39f0850b"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "a30cde0875fd38ee46d50527742990a2"
server_key_iv: "c47b9f6ddb07e045018dc3f13ed05e73"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.157718 28054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:45:46.158349 28069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.158535 28054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:46.158617 28054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "6af69bc7c2fa4962976d266d39f0850b"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "a30cde0875fd38ee46d50527742990a2"
server_key_iv: "c47b9f6ddb07e045018dc3f13ed05e73"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.158685 28054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:46.165776 28054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:46.167927 28054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:46.168046 28054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:46.171952 28054 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:36125
I20260619 02:45:46.171983 28121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:36125 every 8 connection(s)
I20260619 02:45:46.172302 28054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:46.173125 28122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:46.175462 28122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Bootstrap starting.
I20260619 02:45:46.176043 28122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:46.176308 28122 log.cc:824] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:46.176918 28122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: No bootstrap required, opened a new log
I20260619 02:45:46.178457 28122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } }
I20260619 02:45:46.178589 28122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:46.178622 28122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af69bc7c2fa4962976d266d39f0850b, State: Initialized, Role: FOLLOWER
I20260619 02:45:46.178745 28122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } }
I20260619 02:45:46.178804 28122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:46.178833 28122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:46.178907 28122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:46.179394 28122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } }
I20260619 02:45:46.179483 28122 leader_election.cc:302] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af69bc7c2fa4962976d266d39f0850b; no voters:
I20260619 02:45:46.179636 28122 leader_election.cc:288] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:46.179675 28125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:46.179870 28122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:46.180006 28125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 1 LEADER]: Becoming Leader. State: Replica: 6af69bc7c2fa4962976d266d39f0850b, State: Running, Role: LEADER
I20260619 02:45:46.180096 28125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } }
I20260619 02:45:46.180424 28127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6af69bc7c2fa4962976d266d39f0850b. Latest consensus state: current_term: 1 leader_uuid: "6af69bc7c2fa4962976d266d39f0850b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } } }
I20260619 02:45:46.180521 28127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:46.180660 28126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6af69bc7c2fa4962976d266d39f0850b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } } }
I20260619 02:45:46.180769 28126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:46.180706 28134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:46.181255 28134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:46.181703 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 28054
I20260619 02:45:46.181805 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:45:46.181993 19909 external_mini_cluster.cc:1428] Setting key 8926f4225fd712c46cff2f0d5e03ba88
I20260619 02:45:46.182835 28134 catalog_manager.cc:1347] Generated new cluster ID: 145234339ea544998c727a18099c4d27
I20260619 02:45:46.182897 28134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:46.193897 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:46.182408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57152 (local address 127.19.113.126:36125)
0619 02:45:46.182547 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:46.182556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:46.182995 (+ 439us) server_negotiation.cc:407] Connection header received
0619 02:45:46.183141 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:46.183148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:46.183200 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:46.183325 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:46.184096 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.185463 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.186522 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.187640 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.187737 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:46.190427 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:46.190439 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:46.190440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:46.190456 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:46.192377 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:46.192730 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:46.192733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:46.192734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:46.192840 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:46.193003 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:46.193003 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:46.193004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:46.193310 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:46.193449 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:46.193523 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:45:46.193659 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":71,"threads_started":1}
I20260619 02:45:46.195510 28134 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:46.196015 28134 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:46.205193 28134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Generated new TSK 0
I20260619 02:45:46.205443 28134 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:45:46.237524 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36125
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:46.330530 28148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:46.330742 28148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:46.330773 28148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:46.331991 28148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:46.332036 28148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:46.332070 28148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:46.333539 28148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:46.333770 28148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:46.333978 28148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:46.338601 28153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:46.338624 28154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:46.338609 28156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:46.338740 28148 server_base.cc:1034] running on GCE node
I20260619 02:45:46.339093 28148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:46.339323 28148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:46.340469 28148 hybrid_clock.cc:648] HybridClock initialized: now 1781837146340456 us; error 31 us; skew 500 ppm
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:46.343053 28148 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:45:46.343449 28148 webserver.cc:460] Webserver started at http://127.19.113.65:38883/ using document root <none> and password file <none>
I20260619 02:45:46.343648 28148 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:46.343689 28148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:46.343815 28148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:46.344796 28148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "d121dfaef9274a6383c1cb59efb85fb9"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "de3d1fa357d389780eb6d2e137b18f98"
server_key_iv: "a6cd05d6c2d4fe2aada4e9540032d0d6"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.345113 28148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "d121dfaef9274a6383c1cb59efb85fb9"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "de3d1fa357d389780eb6d2e137b18f98"
server_key_iv: "a6cd05d6c2d4fe2aada4e9540032d0d6"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.346283 28148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:46.346932 28163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.347071 28148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:46.347152 28148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "d121dfaef9274a6383c1cb59efb85fb9"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "de3d1fa357d389780eb6d2e137b18f98"
server_key_iv: "a6cd05d6c2d4fe2aada4e9540032d0d6"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.347213 28148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:46.363958 28148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:46.365702 28148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:46.365836 28148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:46.366050 28148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:46.366338 28148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:46.366369 28148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.366398 28148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:46.366422 28148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.371950 28148 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:43577
I20260619 02:45:46.371992 28276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:43577 every 8 connection(s)
I20260619 02:45:46.372304 28148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:46.372857 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 28148
I20260619 02:45:46.372946 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:45:46.373159 19909 external_mini_cluster.cc:1428] Setting key f41735897df9a352249cf8cb1d9ba5b2
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
I20260619 02:45:46.382812 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:46.373057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:33075 (local address 127.19.113.126:36125)
0619 02:45:46.373124 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:46.373127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:46.373334 (+ 207us) server_negotiation.cc:407] Connection header received
0619 02:45:46.373898 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:46.373899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:46.373912 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:46.373981 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:46.375216 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.375716 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.376936 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.377245 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.377319 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:46.379664 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:46.379666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:46.379666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:46.379676 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:46.381159 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:46.381712 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:46.381714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:46.381715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:46.381776 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:46.382068 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:46.382069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:46.382069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:46.382207 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:46.382269 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:46.382617 (+ 348us) server_negotiation.cc:299] Negotiation successful
0619 02:45:46.382713 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:45:46.383037 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:46.372970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:33075)
0619 02:45:46.373200 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:46.373250 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:46.373813 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:46.374069 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:46.374074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:46.374797 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:46.375100 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:46.375107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.375808 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.375809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:46.376834 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:46.376838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.377322 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.377323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:46.377477 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:46.378029 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:46.378037 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:46.379565 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:46.381267 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:46.381275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:46.381277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:46.381641 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:46.381851 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:46.381851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:46.381851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:46.381971 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:46.382260 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:46.382267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:46.382447 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 02:45:46.382566 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:45:46.382762 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260619 02:45:46.383512 28277 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36125
I20260619 02:45:46.383652 28277 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
I20260619 02:45:46.383858 28277 heartbeater.cc:511] Master 127.19.113.126:36125 requested a full tablet report, sending...
I20260619 02:45:46.384354 28086 ts_manager.cc:194] Registered new tserver with Master: d121dfaef9274a6383c1cb59efb85fb9 (127.19.113.65:43577)
I20260619 02:45:46.384953 28086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:33075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:45:46.395344 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36125
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:46.474722 28282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:46.474958 28282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:46.474983 28282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:46.476187 28282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:46.476231 28282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:46.476258 28282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:46.477761 28282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:46.478015 28282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:46.478229 28282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:46.483295 28288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:46.483271 28287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:46.483279 28290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:46.483362 28282 server_base.cc:1034] running on GCE node
I20260619 02:45:46.483642 28282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:46.483848 28282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:46.485002 28282 hybrid_clock.cc:648] HybridClock initialized: now 1781837146484987 us; error 26 us; skew 500 ppm
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:46.487404 28282 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:45:46.487800 28282 webserver.cc:460] Webserver started at http://127.19.113.66:34043/ using document root <none> and password file <none>
I20260619 02:45:46.488031 28282 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:46.488076 28282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:46.488197 28282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:46.489233 28282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "48a29e038652454cac3e57f44237eaf4"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "668b90c628c6c2ab5ea48f6ad08ae005"
server_key_iv: "98bd7c285077af27969dbbd37671fe92"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.489559 28282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "48a29e038652454cac3e57f44237eaf4"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "668b90c628c6c2ab5ea48f6ad08ae005"
server_key_iv: "98bd7c285077af27969dbbd37671fe92"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.490795 28282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:45:46.491526 28297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.491677 28282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:46.491756 28282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "48a29e038652454cac3e57f44237eaf4"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "668b90c628c6c2ab5ea48f6ad08ae005"
server_key_iv: "98bd7c285077af27969dbbd37671fe92"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.491822 28282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:46.515215 28282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:46.517279 28282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:46.517400 28282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:46.517625 28282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:46.517922 28282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:46.517962 28282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.517992 28282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:46.518010 28282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.523366 28282 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:38383
I20260619 02:45:46.523419 28410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:38383 every 8 connection(s)
I20260619 02:45:46.523708 28282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:46.530088 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 28282
I20260619 02:45:46.530185 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:45:46.530385 19909 external_mini_cluster.cc:1428] Setting key 4ca1baec02ece881748ea540faa0ca2f
I20260619 02:45:46.532856 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:46.524414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:47739 (local address 127.19.113.126:36125)
0619 02:45:46.524482 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:46.524484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:46.524690 (+ 206us) server_negotiation.cc:407] Connection header received
0619 02:45:46.525153 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:46.525154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:46.525163 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:46.525218 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:46.526159 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.526649 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.527671 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.527869 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.527963 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:46.530020 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:46.530022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:46.530022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:46.530030 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:46.531385 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:46.531869 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:46.531870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:46.531870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:46.531922 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:46.532157 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:46.532157 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:46.532158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:46.532254 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:46.532298 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:46.532668 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 02:45:46.532760 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:45:46.533006 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:46.524385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:47739)
0619 02:45:46.524561 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:46.524609 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:46.525079 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:46.525272 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:46.525275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:46.525843 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:46.526086 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:46.526091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.526709 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.526710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:46.527561 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:46.527564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.527993 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.527994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:46.528115 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:46.528578 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:46.528586 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:46.529918 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:46.531481 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:46.531490 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:46.531493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:46.531798 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:46.531974 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:46.531975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:46.531975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:46.532105 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:46.532333 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:46.532339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:46.532512 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 02:45:46.532634 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:45:46.532777 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260619 02:45:46.533510 28411 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36125
I20260619 02:45:46.533619 28411 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:46.533797 28411 heartbeater.cc:511] Master 127.19.113.126:36125 requested a full tablet report, sending...
I20260619 02:45:46.534181 28086 ts_manager.cc:194] Registered new tserver with Master: 48a29e038652454cac3e57f44237eaf4 (127.19.113.66:38383)
I20260619 02:45:46.534680 28086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:47739
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:45:46.554836 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36125
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:46.634658 28416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:46.634892 28416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:46.634922 28416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:46.636087 28416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:46.636122 28416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:46.636147 28416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:46.637566 28416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:46.637768 28416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:46.637961 28416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:46.642944 28424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:46.642943 28422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:46.642939 28421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:46.643329 28416 server_base.cc:1034] running on GCE node
I20260619 02:45:46.643553 28416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:46.643793 28416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:46.644958 28416 hybrid_clock.cc:648] HybridClock initialized: now 1781837146644937 us; error 35 us; skew 500 ppm
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:46.647594 28416 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:45:46.648049 28416 webserver.cc:460] Webserver started at http://127.19.113.67:44253/ using document root <none> and password file <none>
I20260619 02:45:46.648270 28416 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:46.648319 28416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:46.648440 28416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:46.649562 28416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "91aadcb7a5ee42399a2266f381876706"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "0be0ccbd9b2a25ac2a3341f54e8e5849"
server_key_iv: "6625bacd65d40880c2f5b36b9b2e2adf"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.649914 28416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "91aadcb7a5ee42399a2266f381876706"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "0be0ccbd9b2a25ac2a3341f54e8e5849"
server_key_iv: "6625bacd65d40880c2f5b36b9b2e2adf"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.651247 28416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:45:46.651916 28431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.652067 28416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.652148 28416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "91aadcb7a5ee42399a2266f381876706"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "0be0ccbd9b2a25ac2a3341f54e8e5849"
server_key_iv: "6625bacd65d40880c2f5b36b9b2e2adf"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.652211 28416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:46.673952 28416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:46.676074 28416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:46.676206 28416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:46.676425 28416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:46.676754 28416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:46.676792 28416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.676821 28416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:46.676838 28416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.682672 28416 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:38275
I20260619 02:45:46.682734 28544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:38275 every 8 connection(s)
I20260619 02:45:46.683055 28416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:45:46.689900 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 28416
I20260619 02:45:46.690026 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:45:46.690223 19909 external_mini_cluster.cc:1428] Setting key 21cae697b1000f8600196bdf64a47263
I20260619 02:45:46.692427 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:46.683697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:54235 (local address 127.19.113.126:36125)
0619 02:45:46.683738 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:46.683739 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:46.683954 (+ 215us) server_negotiation.cc:407] Connection header received
0619 02:45:46.684402 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:46.684403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:46.684412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:46.684482 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:46.685475 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.685987 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.686995 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.687172 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.687220 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:46.689263 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:46.689264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:46.689265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:46.689274 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:46.690750 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:46.691231 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:46.691233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:46.691233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:46.691360 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:46.691641 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:46.691642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:46.691642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:46.691768 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:46.691858 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:46.692224 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 02:45:46.692308 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:45:46.692718 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:46.683676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:54235)
0619 02:45:46.683826 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:46.683876 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:46.684330 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:46.684544 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:46.684548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:46.685168 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:46.685377 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:46.685382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.686072 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.686073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:46.686926 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:46.686929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:46.687228 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:46.687228 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:46.687324 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:46.687756 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:46.687764 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:46.689147 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:46.690823 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:46.690831 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:46.690833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:46.691167 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:46.691454 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:46.691455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:46.691456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:46.691580 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:46.691843 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:46.691850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:46.692020 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 02:45:46.692140 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 02:45:46.692325 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260619 02:45:46.693228 28545 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36125
I20260619 02:45:46.693334 28545 heartbeater.cc:464] Registering TS with master...
I20260619 02:45:46.693540 28545 heartbeater.cc:511] Master 127.19.113.126:36125 requested a full tablet report, sending...
I20260619 02:45:46.693940 28086 ts_manager.cc:194] Registered new tserver with Master: 91aadcb7a5ee42399a2266f381876706 (127.19.113.67:38275)
I20260619 02:45:46.694483 28086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:54235
I20260619 02:45:46.702118 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab.
Entry for principal oryx/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab.
WARNING: no policy specified for oryx/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab.
Entry for principal oryx/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab.
WARNING: no policy specified for oryx/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab.
Entry for principal oryx/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab.
I20260619 02:45:46.770202 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 28148
I20260619 02:45:46.774518 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 28282
I20260619 02:45:46.778136 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 28416
I20260619 02:45:46.782018 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 28054
I20260619 02:45:46.786382 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36125
--webserver_interface=127.19.113.126
--webserver_port=40769
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:36125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:46.866519 28554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:46.866739 28554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:46.866770 28554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:46.867955 28554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:46.868003 28554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:46.868022 28554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:46.868043 28554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:46.868058 28554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:46.869583 28554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:36125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=40769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:46.869882 28554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:46.870116 28554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:46.874850 28559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:46.874828 28562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:46.874858 28554 server_base.cc:1034] running on GCE node
W20260619 02:45:46.874832 28560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:46.875327 28554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:46.875615 28554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:46.876806 28554 hybrid_clock.cc:648] HybridClock initialized: now 1781837146876791 us; error 24 us; skew 500 ppm
Jun 19 02:45:46 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:46.879194 28554 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:45:46.879673 28554 webserver.cc:460] Webserver started at http://127.19.113.126:40769/ using document root <none> and password file <none>
I20260619 02:45:46.879920 28554 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:46.879994 28554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:46.881703 28554 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:46.882433 28569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:46.882594 28554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:46.882675 28554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "6af69bc7c2fa4962976d266d39f0850b"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "a30cde0875fd38ee46d50527742990a2"
server_key_iv: "c47b9f6ddb07e045018dc3f13ed05e73"
server_key_version: "encryptionkey@0"
I20260619 02:45:46.883067 28554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:46.897665 28554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:46.899560 28554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:46.899686 28554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:46.903620 28554 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:36125
I20260619 02:45:46.903666 28621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:36125 every 8 connection(s)
I20260619 02:45:46.904212 28554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:46.906111 28622 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:45:46.906725 28622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Bootstrap starting.
I20260619 02:45:46.910987 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 28554
I20260619 02:45:46.911259 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:43577
--local_ip_for_outbound_sockets=127.19.113.65
--tserver_master_addrs=127.19.113.126:36125
--webserver_port=38883
--webserver_interface=127.19.113.65
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.113.65
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:45:46.914557 28622 log.cc:824] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:46.916204 28622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:45:46.916452 28622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Bootstrap complete.
I20260619 02:45:46.918558 28622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } }
I20260619 02:45:46.918792 28622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af69bc7c2fa4962976d266d39f0850b, State: Initialized, Role: FOLLOWER
I20260619 02:45:46.918975 28622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } }
I20260619 02:45:46.919054 28622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:46.919099 28622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:46.919143 28622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:45:46.920092 28622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } }
I20260619 02:45:46.920186 28622 leader_election.cc:302] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af69bc7c2fa4962976d266d39f0850b; no voters:
I20260619 02:45:46.920382 28622 leader_election.cc:288] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:45:46.920434 28626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:45:46.920545 28626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [term 2 LEADER]: Becoming Leader. State: Replica: 6af69bc7c2fa4962976d266d39f0850b, State: Running, Role: LEADER
I20260619 02:45:46.920652 28622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:46.920629 28626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } }
I20260619 02:45:46.920888 28626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6af69bc7c2fa4962976d266d39f0850b. Latest consensus state: current_term: 2 leader_uuid: "6af69bc7c2fa4962976d266d39f0850b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } } }
I20260619 02:45:46.920962 28626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:46.920912 28627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6af69bc7c2fa4962976d266d39f0850b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af69bc7c2fa4962976d266d39f0850b" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36125 } } }
I20260619 02:45:46.921114 28627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:46.921164 28630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:46.921686 28630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:46.921845 28630 catalog_manager.cc:1259] Loaded cluster ID: 145234339ea544998c727a18099c4d27
I20260619 02:45:46.921887 28630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:46.923087 28630 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:46.923285 28630 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6af69bc7c2fa4962976d266d39f0850b: Loaded TSK: 0
I20260619 02:45:46.923501 28630 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:45:47.011390 28624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:47.011605 28624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:47.011627 28624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:47.012818 28624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:47.012861 28624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:47.012887 28624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:45:47.014313 28624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.65.keytab
--principal=oryx/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:43577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=38883
--tserver_master_addrs=127.19.113.126:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:47.014533 28624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:47.014752 28624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:47.019316 28651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:47.019316 28649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:47.019397 28648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:47.019608 28624 server_base.cc:1034] running on GCE node
I20260619 02:45:47.019786 28624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:47.020036 28624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:47.021181 28624 hybrid_clock.cc:648] HybridClock initialized: now 1781837147021167 us; error 32 us; skew 500 ppm
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837147, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:47.023551 28624 init.cc:377] Logged in from keytab as oryx/127.19.113.65@KRBTEST.COM (short username oryx)
I20260619 02:45:47.023986 28624 webserver.cc:460] Webserver started at http://127.19.113.65:38883/ using document root <none> and password file <none>
I20260619 02:45:47.024187 28624 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:47.024248 28624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:47.025627 28624 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:47.026207 28658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.026372 28624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:47.026471 28624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "d121dfaef9274a6383c1cb59efb85fb9"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "de3d1fa357d389780eb6d2e137b18f98"
server_key_iv: "a6cd05d6c2d4fe2aada4e9540032d0d6"
server_key_version: "encryptionkey@0"
I20260619 02:45:47.026978 28624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:47.043965 28624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:47.046344 28624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:47.046505 28624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:47.046785 28624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:47.047211 28624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:47.047358 28624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.047420 28624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:47.047447 28624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.054770 28624 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:43577
I20260619 02:45:47.054811 28771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:43577 every 8 connection(s)
I20260619 02:45:47.055285 28624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:45:47.056377 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 28624
I20260619 02:45:47.056650 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:38383
--local_ip_for_outbound_sockets=127.19.113.66
--tserver_master_addrs=127.19.113.126:36125
--webserver_port=34043
--webserver_interface=127.19.113.66
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.113.66
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:47.067742 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.056080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:49739)
0619 02:45:47.057828 (+ 1748us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.057882 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.058528 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.059218 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.059222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.059994 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.060282 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.060288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.061999 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.062000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.063281 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.063284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.064073 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.064074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.064215 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.064819 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.064830 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.067536 (+ 2706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1699,"thread_start_us":86,"threads_started":1}
W20260619 02:45:47.067940 28772 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:36125 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.113.126:36125: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
W20260619 02:45:47.068166 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.056116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:49739 (local address 127.19.113.126:36125)
0619 02:45:47.056299 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.056308 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.058043 (+ 1735us) server_negotiation.cc:407] Connection header received
0619 02:45:47.058928 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.058936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.059005 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.059143 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.060417 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.061883 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.063371 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.063973 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.064087 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.067677 (+ 3590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:49739: BlockingRecv error: recv got EOF from 127.19.113.65:49739 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:47.073575 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.068147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:33161)
0619 02:45:47.068458 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.068470 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.068519 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.068657 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.068658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.068799 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.068850 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.068852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.069693 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.069694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.070494 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.070497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.070962 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.070963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.071025 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.071483 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.071488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.073502 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294}
W20260619 02:45:47.073846 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.068254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:33161 (local address 127.19.113.126:36125)
0619 02:45:47.068462 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.068464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.068516 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:45:47.068570 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.068571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.068583 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.068649 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.068908 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.069614 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.070569 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.070858 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.070958 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.073787 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:33161: BlockingRecv error: recv got EOF from 127.19.113.65:33161 (error 108)
Metrics: {"mutex_wait_us":154,"server-negotiator.queue_time_us":188}
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:47.078982 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.073821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:39181)
0619 02:45:47.074043 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.074052 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.074097 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.074223 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.074223 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.074355 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.074390 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.074392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.075277 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.075278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.076088 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.076090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.076510 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.076510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.076564 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.076998 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.077003 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.078907 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207}
W20260619 02:45:47.079118 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.073901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:39181 (local address 127.19.113.126:36125)
0619 02:45:47.074033 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.074036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.074100 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:45:47.074149 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.074149 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.074160 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.074210 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.074453 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.075204 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.076157 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.076448 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.076504 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.079054 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:39181: BlockingRecv error: recv got EOF from 127.19.113.65:39181 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":96,"threads_started":1}
W20260619 02:45:47.079401 28772 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:45:47.141211 28776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:47.141407 28776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:47.141435 28776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:47.142585 28776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:47.142625 28776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:47.142649 28776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:45:47.144045 28776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.66.keytab
--principal=oryx/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:38383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=34043
--tserver_master_addrs=127.19.113.126:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:47.144256 28776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:47.144462 28776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:47.149060 28785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:47.149080 28783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:47.149060 28782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:47.149235 28776 server_base.cc:1034] running on GCE node
I20260619 02:45:47.149456 28776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:47.149672 28776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:47.150827 28776 hybrid_clock.cc:648] HybridClock initialized: now 1781837147150816 us; error 28 us; skew 500 ppm
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837147, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:47.153288 28776 init.cc:377] Logged in from keytab as oryx/127.19.113.66@KRBTEST.COM (short username oryx)
I20260619 02:45:47.153714 28776 webserver.cc:460] Webserver started at http://127.19.113.66:34043/ using document root <none> and password file <none>
I20260619 02:45:47.153910 28776 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:47.153970 28776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:47.155234 28776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:47.155745 28792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.155879 28776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:45:47.155954 28776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "48a29e038652454cac3e57f44237eaf4"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "668b90c628c6c2ab5ea48f6ad08ae005"
server_key_iv: "98bd7c285077af27969dbbd37671fe92"
server_key_version: "encryptionkey@0"
I20260619 02:45:47.156275 28776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:47.190593 28776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:47.192721 28776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:47.192859 28776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:47.193077 28776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:47.193354 28776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:47.193387 28776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.193414 28776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:47.193437 28776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.198740 28776 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:38383
I20260619 02:45:47.198781 28905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:38383 every 8 connection(s)
I20260619 02:45:47.199110 28776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:45:47.201968 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 28776
I20260619 02:45:47.202216 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:38275
--local_ip_for_outbound_sockets=127.19.113.67
--tserver_master_addrs=127.19.113.126:36125
--webserver_port=44253
--webserver_interface=127.19.113.67
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.113.67
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:47.206368 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.199750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:42689)
0619 02:45:47.199903 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.199941 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.200414 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.200655 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.200659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.201202 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.201382 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.201386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.202058 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.202059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.202956 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.202958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.203348 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.203349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.203488 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.203963 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.203974 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.206218 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
W20260619 02:45:47.206405 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.199764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:42689 (local address 127.19.113.126:36125)
0619 02:45:47.199820 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.199822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.200022 (+ 200us) server_negotiation.cc:407] Connection header received
0619 02:45:47.200512 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.200513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.200530 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.200577 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.201475 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.201983 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.203031 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.203219 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.203280 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.206340 (+ 3060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:42689: BlockingRecv error: recv got EOF from 127.19.113.66:42689 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 02:45:47.206570 28906 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:36125 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.113.126:36125: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:47.211158 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.206770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:60181)
0619 02:45:47.206891 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.206901 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.206951 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.207065 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.207066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.207205 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.207261 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.207262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.207847 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.207848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.208617 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.208619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.208934 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.208935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.209002 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.209356 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.209362 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.211086 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260619 02:45:47.211210 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.206825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:60181 (local address 127.19.113.126:36125)
0619 02:45:47.206901 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.206903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.206946 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:45:47.207007 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.207008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.207015 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.207059 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.207318 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.207778 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.208691 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.208868 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.208917 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.211160 (+ 2243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:60181: BlockingRecv error: recv got EOF from 127.19.113.66:60181 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:47.215863 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.211403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:33445)
0619 02:45:47.211444 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.211454 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.211488 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.211617 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.211618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.211751 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.211796 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.211797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.212388 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.212389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.213217 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.213219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.213530 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.213531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.213582 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.213920 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.213925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.215737 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:45:47.215893 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.211449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:33445 (local address 127.19.113.126:36125)
0619 02:45:47.211490 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.211492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.211500 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:47.211538 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.211538 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.211546 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.211589 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.211857 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.212320 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.213293 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.213472 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.213519 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.215839 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:33445: BlockingRecv error: recv got EOF from 127.19.113.66:33445 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 02:45:47.216135 28906 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:45:47.284387 28909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:47.284601 28909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:47.284623 28909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:47.285847 28909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:47.285892 28909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:47.285916 28909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:45:47.287437 28909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/krb5kdc/oryx_127.19.113.67.keytab
--principal=oryx/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:38275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=44253
--tserver_master_addrs=127.19.113.126:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:47.287660 28909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:47.287885 28909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:47.292572 28915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:47.292609 28909 server_base.cc:1034] running on GCE node
W20260619 02:45:47.292577 28914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:47.292593 28917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:47.293154 28909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:47.293407 28909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:47.294560 28909 hybrid_clock.cc:648] HybridClock initialized: now 1781837147294547 us; error 23 us; skew 500 ppm
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837147, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:47.297008 28909 init.cc:377] Logged in from keytab as oryx/127.19.113.67@KRBTEST.COM (short username oryx)
I20260619 02:45:47.297484 28909 webserver.cc:460] Webserver started at http://127.19.113.67:44253/ using document root <none> and password file <none>
I20260619 02:45:47.297686 28909 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:47.297747 28909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:47.299144 28909 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:47.299710 28924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.299903 28909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.299995 28909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "91aadcb7a5ee42399a2266f381876706"
format_stamp: "Formatted at 2026-06-19 02:45:46 on dist-test-slave-zqpb"
server_key: "0be0ccbd9b2a25ac2a3341f54e8e5849"
server_key_iv: "6625bacd65d40880c2f5b36b9b2e2adf"
server_key_version: "encryptionkey@0"
I20260619 02:45:47.300372 28909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:47.314584 28909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:47.316457 28909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:47.316588 28909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:47.316924 28909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:45:47.317250 28909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:45:47.317283 28909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.317312 28909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:45:47.317337 28909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:47.323092 28909 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:38275
I20260619 02:45:47.323144 29037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:38275 every 8 connection(s)
I20260619 02:45:47.323457 28909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:45:47.327486 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 28909
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:45:47.330688 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.324098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:46119 (local address 127.19.113.126:36125)
0619 02:45:47.324160 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.324163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.324366 (+ 203us) server_negotiation.cc:407] Connection header received
0619 02:45:47.324852 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.324853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.324865 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.324915 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.325789 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.326279 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.327283 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.327481 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.327535 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.330643 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:46119: BlockingRecv error: recv got EOF from 127.19.113.67:46119 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:45:47.330780 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.324080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:46119)
0619 02:45:47.324241 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.324283 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.324754 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.324979 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.324982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.325515 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.325699 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.325704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.326371 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.326372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.327204 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.327207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.327548 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.327549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.327675 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.328228 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.328239 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.330574 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
W20260619 02:45:47.331058 29038 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:36125 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.113.126:36125: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:47.335343 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.327984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57158 (local address 127.19.113.126:36125)
0619 02:45:47.328039 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.328041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.328119 (+ 78us) server_negotiation.cc:407] Connection header received
0619 02:45:47.328199 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.328200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.328211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.328261 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.328984 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.329502 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.330289 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.330531 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.330586 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.332052 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:47.332056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:47.332057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:47.332069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:47.333780 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:47.334146 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:47.334151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:47.334151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:47.334272 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:47.334469 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:47.334469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:47.334470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:47.334774 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:47.334912 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:47.335124 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 02:45:47.335249 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:45:47.335693 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.331295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:40047)
0619 02:45:47.331339 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.331348 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.331383 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.331511 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.331512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.331639 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.331698 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.331700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.332467 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.332468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.333162 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.333164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.333573 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.333574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.333628 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.333952 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.333956 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.335631 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 02:45:47.335757 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.331367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:40047 (local address 127.19.113.126:36125)
0619 02:45:47.331410 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.331412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.331418 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:45:47.331437 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.331438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.331447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.331491 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.331824 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.332388 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.333237 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.333499 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.333568 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.335710 (+ 2142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:40047: BlockingRecv error: recv got EOF from 127.19.113.67:40047 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:47 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:47.339448 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:47.335924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:51687)
0619 02:45:47.335958 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:47.335966 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:47.335998 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:47.336102 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:47.336103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:47.336213 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:47.336238 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.336239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.336788 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.336789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.337259 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:47.337260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.337564 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.337564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:47.337612 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.337960 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:47.337964 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:47.339393 (+ 1429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 02:45:47.339548 28775 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:47.335962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:51687 (local address 127.19.113.126:36125)
0619 02:45:47.335993 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:47.335994 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:47.336001 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:47.336040 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:47.336041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:47.336049 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:47.336098 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:47.336296 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.336727 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.337324 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:47.337512 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:47.337561 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:47.339494 (+ 1933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:51687: BlockingRecv error: recv got EOF from 127.19.113.67:51687 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:45:47.339610 29038 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 02:45:48 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:48 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:48.084079 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:48.079792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:56175)
0619 02:45:48.079914 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:48.079933 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:48.079983 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:48.080138 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:48.080139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:48.080288 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:48.080338 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:48.080340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.080975 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.080975 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:48.081545 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:48.081547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.081906 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.081907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:48.081952 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:48.082334 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:48.082337 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:48.084010 (+ 1673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
W20260619 02:45:48.084175 29043 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:48.079884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:56175 (local address 127.19.113.126:36125)
0619 02:45:48.080012 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:48.080014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:48.080024 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:48.080049 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:48.080050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:48.080059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:48.080127 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:48.080395 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.080917 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.081620 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.081817 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.081903 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:48.084099 (+ 2196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:56175: BlockingRecv error: recv got EOF from 127.19.113.65:56175 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
Jun 19 02:45:48 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:48 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:48.221290 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:48.216589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:56887)
0619 02:45:48.216710 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:48.216721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:48.216775 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:48.216905 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:48.216905 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:48.217089 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:48.217147 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:48.217150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.217875 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.217876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:48.218452 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:48.218454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.218819 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.218820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:48.218902 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:48.219364 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:48.219369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:48.221214 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
W20260619 02:45:48.221380 29043 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:48.216632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:56887 (local address 127.19.113.126:36125)
0619 02:45:48.216692 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:48.216694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:48.216772 (+ 78us) server_negotiation.cc:407] Connection header received
0619 02:45:48.216819 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:48.216821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:48.216833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:48.216884 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:48.217261 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.217793 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.218543 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.218746 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.218810 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:48.221321 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:56887: BlockingRecv error: recv got EOF from 127.19.113.66:56887 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 19 02:45:48 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:48 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:48.344326 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:48.340004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:47617)
0619 02:45:48.340163 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:48.340172 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:48.340237 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:48.340358 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:48.340359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:48.340522 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:48.340570 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:48.340572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.341213 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.341213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:48.341759 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:48.341761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.342138 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.342138 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:48.342181 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:48.342579 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:48.342583 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:48.344260 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
W20260619 02:45:48.344430 29043 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:48.340083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:47617 (local address 127.19.113.126:36125)
0619 02:45:48.340148 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:48.340150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:48.340230 (+ 80us) server_negotiation.cc:407] Connection header received
0619 02:45:48.340291 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:48.340291 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:48.340304 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:48.340348 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:48.340644 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.341143 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.341833 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:48.342072 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:48.342125 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:48.344368 (+ 2243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:47617: BlockingRecv error: recv got EOF from 127.19.113.67:47617 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 02:45:49 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:49 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:49.089087 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:49.084582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:42697)
0619 02:45:49.084735 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:49.084746 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:49.084801 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:49.084956 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:49.084957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:49.085153 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:49.085211 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:49.085213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.085939 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.085940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:49.086462 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:49.086463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.086819 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.086819 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:49.086905 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:49.087335 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:49.087341 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:49.089028 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
W20260619 02:45:49.089195 29047 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:49.084651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:42697 (local address 127.19.113.126:36125)
0619 02:45:49.084776 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:49.084778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:49.084788 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:49.084859 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:49.084860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:49.084873 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:49.084937 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:49.085280 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.085856 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.086537 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.086757 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.086817 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:49.089126 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:42697: BlockingRecv error: recv got EOF from 127.19.113.65:42697 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":47,"threads_started":1}
Jun 19 02:45:49 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:49 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:49.226413 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:49.221782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:41115)
0619 02:45:49.221948 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:49.221958 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:49.222017 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:49.222192 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:49.222192 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:49.222348 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:49.222395 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:49.222397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.223062 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.223063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:49.223613 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:49.223615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.224068 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.224070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:49.224137 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:49.224635 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:49.224641 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:49.226335 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
W20260619 02:45:49.226503 29047 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:49.221843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:41115 (local address 127.19.113.126:36125)
0619 02:45:49.221889 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:49.221890 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:49.222014 (+ 124us) server_negotiation.cc:407] Connection header received
0619 02:45:49.222099 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:49.222100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:49.222113 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:49.222173 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:49.222469 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.222992 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.223694 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.223950 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.224023 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:49.226448 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:41115: BlockingRecv error: recv got EOF from 127.19.113.66:41115 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 02:45:49 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:49 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:49.349078 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:49.344804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:55663)
0619 02:45:49.344973 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:49.344988 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:49.345055 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:49.345202 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:49.345202 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:49.345357 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:49.345404 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:49.345405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.346006 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.346008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:49.346548 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:49.346549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.346853 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.346854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:49.346939 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:49.347379 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:49.347384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:49.349019 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
W20260619 02:45:49.349169 29047 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:49.345009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:55663 (local address 127.19.113.126:36125)
0619 02:45:49.345092 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:49.345093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:49.345103 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:49.345124 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:49.345124 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:49.345137 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:49.345186 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:49.345460 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.345936 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.346621 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:49.346798 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:49.346840 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:49.349109 (+ 2269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:55663: BlockingRecv error: recv got EOF from 127.19.113.67:55663 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 02:45:50 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:50 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:50.094059 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:50.089556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:57163)
0619 02:45:50.089724 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:50.089735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:50.089792 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:50.089925 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:50.089926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:50.090080 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:50.090131 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:50.090132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.090853 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.090854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:50.091426 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:50.091429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.091763 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.091764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:50.091814 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:50.092235 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:50.092239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:50.094001 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":61,"threads_started":1}
W20260619 02:45:50.094165 29051 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:50.089634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:57163 (local address 127.19.113.126:36125)
0619 02:45:50.089807 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:50.089809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:50.089817 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:50.089839 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:50.089840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:50.089850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:50.089912 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:50.090197 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.090754 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.091503 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.091703 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.091760 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:50.094096 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:57163: BlockingRecv error: recv got EOF from 127.19.113.65:57163 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":52,"threads_started":1}
Jun 19 02:45:50 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:50 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:50.231235 29052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:50.226961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:54779)
0619 02:45:50.227097 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:50.227108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:50.227154 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:50.227272 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:50.227273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:50.227451 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:50.227499 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:50.227500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.228147 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.228148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:50.228683 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:50.228684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.229022 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.229022 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:50.229061 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:50.229477 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:50.229481 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:50.231155 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
W20260619 02:45:50.231396 29051 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:50.227048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:54779 (local address 127.19.113.126:36125)
0619 02:45:50.227138 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:50.227140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:50.227149 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:50.227201 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:50.227201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:50.227212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:50.227256 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:50.227556 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.228057 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.228750 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.228963 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.229010 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:50.231330 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:54779: BlockingRecv error: recv got EOF from 127.19.113.66:54779 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 02:45:50 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:50 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:50.354079 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:50.349572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:54837)
0619 02:45:50.349695 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:50.349718 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:50.349786 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:50.349946 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:50.349947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:50.350093 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:50.350140 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:50.350142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.350802 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.350804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:50.351479 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:50.351482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.351824 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.351825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:50.351873 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:50.352301 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:50.352305 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:50.354018 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
W20260619 02:45:50.354207 29051 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:50.349668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:54837 (local address 127.19.113.126:36125)
0619 02:45:50.349732 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:50.349735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:50.349769 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:45:50.349853 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:50.349854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:50.349867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:50.349914 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:50.350195 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.350696 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.351550 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:50.351747 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:50.351799 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:50.354142 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:54837: BlockingRecv error: recv got EOF from 127.19.113.67:54837 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Jun 19 02:45:51 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:51 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:51.098946 29054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:51.094488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:52005)
0619 02:45:51.094622 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:51.094633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:51.094698 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:51.094891 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:51.094892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:51.095053 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:51.095100 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:51.095102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.095732 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.095733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:51.096265 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:51.096268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.096627 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.096628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:51.096684 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:51.097118 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:51.097123 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:51.098846 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
W20260619 02:45:51.099083 29055 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:51.094517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:52005 (local address 127.19.113.126:36125)
0619 02:45:51.094650 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:51.094652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:51.094679 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:45:51.094766 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:51.094767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:51.094779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:51.094844 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:51.095165 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.095669 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.096337 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.096535 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.096588 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:51.099011 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:52005: BlockingRecv error: recv got EOF from 127.19.113.65:52005 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
Jun 19 02:45:51 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:51 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:51.241572 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:51.231805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:38181)
0619 02:45:51.231946 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:51.231958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:51.232020 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:51.232199 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:51.232200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:51.232363 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:51.232413 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:51.232415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.233081 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.233083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:51.233632 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:51.233633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.234024 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.234026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:51.234078 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:51.239772 (+ 5694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:51.239777 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:51.241511 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":99,"threads_started":1}
W20260619 02:45:51.241671 29055 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:51.231884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:38181 (local address 127.19.113.126:36125)
0619 02:45:51.231968 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:51.231970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:51.232008 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:45:51.232076 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:51.232077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:51.232088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:51.232145 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:51.232482 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.232991 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.233716 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.233932 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.233986 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:51.241613 (+ 7627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:38181: BlockingRecv error: recv got EOF from 127.19.113.66:38181 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 02:45:51 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:51 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:51.359288 29057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:51.354618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:60403)
0619 02:45:51.354791 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:51.354803 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:51.354898 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:51.355094 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:51.355095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:51.355246 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:51.355293 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:51.355295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.355937 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.355938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:51.356593 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:51.356596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.356968 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.356969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:51.357011 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:51.357435 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:51.357439 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:51.359207 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":63,"threads_started":1}
W20260619 02:45:51.359337 29055 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:51.354722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:60403 (local address 127.19.113.126:36125)
0619 02:45:51.354789 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:51.354791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:51.354891 (+ 100us) server_negotiation.cc:407] Connection header received
0619 02:45:51.354961 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:51.354962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:51.354975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:51.355024 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:51.355352 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.355872 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.356669 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:51.356885 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:51.356937 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:51.359280 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:60403: BlockingRecv error: recv got EOF from 127.19.113.67:60403 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 02:45:52 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:52 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:52.103928 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:52.099434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:58597)
0619 02:45:52.099582 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:52.099601 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:52.099665 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:52.099819 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:52.099821 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:52.099982 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:52.100032 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:52.100034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.100654 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.100654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:52.101212 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:52.101214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.101556 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.101557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:52.101602 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:52.102014 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:52.102018 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:52.103811 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
W20260619 02:45:52.104009 29059 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:52.099520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:58597 (local address 127.19.113.126:36125)
0619 02:45:52.099613 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:52.099615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:52.099663 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:45:52.099723 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:52.099724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:52.099734 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:52.099788 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:52.100092 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.100596 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.101282 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.101491 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.101540 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:52.103938 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:58597: BlockingRecv error: recv got EOF from 127.19.113.65:58597 (error 108)
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":45,"threads_started":1}
Jun 19 02:45:52 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:52 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:52.246697 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:52.242084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:44403)
0619 02:45:52.242227 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:52.242241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:52.242310 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:52.242487 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:52.242489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:52.242645 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:52.242694 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:52.242696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.243402 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.243403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:52.243932 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:52.243934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.244313 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.244314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:52.244355 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:52.244762 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:52.244767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:52.246605 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
W20260619 02:45:52.246930 29059 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:52.242252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:44403 (local address 127.19.113.126:36125)
0619 02:45:52.242340 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:52.242341 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:52.242352 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:52.242378 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:52.242379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:52.242390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:52.242447 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:52.242773 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.243326 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.244024 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.244224 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.244291 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:52.246780 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:44403: BlockingRecv error: recv got EOF from 127.19.113.66:44403 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 02:45:52 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:52 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:52.364305 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:52.359964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:60215)
0619 02:45:52.360114 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:52.360127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:52.360200 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:52.360335 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:52.360336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:52.360489 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:52.360538 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:52.360539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.361157 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.361158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:52.361682 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:52.361683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.362011 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.362012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:52.362059 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:52.362528 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:52.362533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:52.364245 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
W20260619 02:45:52.364425 29059 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:52.360048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:60215 (local address 127.19.113.126:36125)
0619 02:45:52.360126 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:52.360128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:52.360190 (+ 62us) server_negotiation.cc:407] Connection header received
0619 02:45:52.360263 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:52.360264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:52.360276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:52.360325 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:52.360588 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.361089 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.361755 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:52.361944 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:52.361994 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:52.364361 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:60215: BlockingRecv error: recv got EOF from 127.19.113.67:60215 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 02:45:53 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:53 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:53.109344 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:53.104420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:53541)
0619 02:45:53.104531 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:53.104542 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:53.104590 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:53.104749 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:53.104750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:53.104924 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:53.104973 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:53.104975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.105641 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.105642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:53.106187 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:53.106189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.106695 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.106696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:53.106796 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:53.107268 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:53.107273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:53.109280 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
W20260619 02:45:53.109481 29063 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:53.104474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:53541 (local address 127.19.113.126:36125)
0619 02:45:53.104620 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:53.104622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:53.104630 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:53.104653 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:53.104654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:53.104663 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:53.104728 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:53.105041 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.105552 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.106385 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.106598 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.106658 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:53.109403 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:53541: BlockingRecv error: recv got EOF from 127.19.113.65:53541 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
Jun 19 02:45:53 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:53 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:53.251611 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:53.247227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:54459)
0619 02:45:53.247377 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:53.247390 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:53.247446 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:53.247564 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:53.247564 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:53.247715 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:53.247764 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:53.247765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.248389 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.248389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:53.248934 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:53.248935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.249266 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.249267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:53.249303 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:53.249767 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:53.249771 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:53.251534 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
W20260619 02:45:53.251693 29063 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:53.247374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:54459 (local address 127.19.113.126:36125)
0619 02:45:53.247435 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:53.247437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:53.247445 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:53.247501 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:53.247501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:53.247512 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:53.247548 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:53.247825 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.248323 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.249009 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.249207 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.249262 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:53.251639 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:54459: BlockingRecv error: recv got EOF from 127.19.113.66:54459 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 02:45:53 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:53 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:53.368875 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:53.364783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:46603)
0619 02:45:53.364931 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:53.364941 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:53.364997 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:53.365110 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:53.365111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:53.365262 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:53.365310 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:53.365311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.365916 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.365917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:53.366430 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:53.366431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.366710 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.366710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:53.366746 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:53.367197 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:53.367203 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:53.368817 (+ 1614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
W20260619 02:45:53.368973 29063 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:53.364846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:46603 (local address 127.19.113.126:36125)
0619 02:45:53.364935 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:53.364938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:53.364997 (+ 59us) server_negotiation.cc:407] Connection header received
0619 02:45:53.365040 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:53.365040 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:53.365054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:53.365100 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:53.365362 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.365857 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.366489 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:53.366660 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:53.366702 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:53.368912 (+ 2210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:46603: BlockingRecv error: recv got EOF from 127.19.113.67:46603 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 02:45:54 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:54 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:54.114308 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:54.109914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:39827)
0619 02:45:54.110049 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:54.110061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:54.110115 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:54.110278 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:54.110279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:54.110483 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:54.110533 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:54.110536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.111201 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.111202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:54.111770 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:54.111773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.112076 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.112077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:54.112126 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:54.112527 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:54.112532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:54.114249 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
W20260619 02:45:54.114419 29067 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:54.109991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:39827 (local address 127.19.113.126:36125)
0619 02:45:54.110099 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:54.110101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:54.110110 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:54.110178 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:54.110178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:54.110191 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:54.110261 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:54.110599 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.111120 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.111829 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.112015 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.112057 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:54.114336 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:39827: BlockingRecv error: recv got EOF from 127.19.113.65:39827 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":51,"threads_started":1}
Jun 19 02:45:54 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:54 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:54.256498 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:54.252137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:35033)
0619 02:45:54.252299 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:54.252312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:54.252381 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:54.252511 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:54.252512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:54.252668 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:54.252772 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:54.252774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.253407 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.253409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:54.253995 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:54.253997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.254310 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.254311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:54.254357 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:54.254787 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:54.254792 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:54.256441 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
W20260619 02:45:54.256590 29067 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:54.252275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:35033 (local address 127.19.113.126:36125)
0619 02:45:54.252337 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:54.252339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:54.252365 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:45:54.252436 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:54.252437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:54.252449 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:54.252486 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:54.252847 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.253331 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.254062 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.254246 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.254291 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:54.256531 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:35033: BlockingRecv error: recv got EOF from 127.19.113.66:35033 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 02:45:54 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:54 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:54.373961 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:54.369388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:46911)
0619 02:45:54.369530 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:54.369548 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:54.369607 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:54.369725 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:54.369726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:54.369895 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:54.369946 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:54.369947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.370608 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.370609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:54.371190 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:54.371193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.371573 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.371574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:54.371627 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:54.372114 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:54.372119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:54.373894 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260619 02:45:54.374061 29067 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:54.369502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:46911 (local address 127.19.113.126:36125)
0619 02:45:54.369567 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:54.369569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:54.369591 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:45:54.369654 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:54.369654 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:54.369667 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:54.369719 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:54.370001 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.370535 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.371286 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:54.371473 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:54.371531 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:54.373994 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:46911: BlockingRecv error: recv got EOF from 127.19.113.67:46911 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 02:45:55 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:55 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:55.119043 29070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:55.114714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:58625)
0619 02:45:55.114837 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:55.114847 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:55.114930 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:55.115081 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:55.115082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:55.115232 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:55.115280 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:55.115282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.115909 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.115909 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:55.116460 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:55.116462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.116757 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.116758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:55.116793 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:55.117198 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:55.117202 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:55.118954 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
W20260619 02:45:55.119107 29071 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:55.114856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:58625 (local address 127.19.113.126:36125)
0619 02:45:55.114971 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:55.114973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:55.114980 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:55.115000 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:55.115000 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:55.115010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:55.115066 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:55.115346 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.115851 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.116520 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.116709 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.116751 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:55.119046 (+ 2295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:58625: BlockingRecv error: recv got EOF from 127.19.113.65:58625 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
Jun 19 02:45:55 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:55 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:55.261358 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:55.256950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:50791)
0619 02:45:55.257073 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:55.257084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:55.257134 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:55.257258 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:55.257259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:55.257420 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:55.257468 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:55.257470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.258275 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.258276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:55.258792 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:55.258794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.259124 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.259125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:55.259180 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:55.259562 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:55.259565 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:55.261265 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
W20260619 02:45:55.261452 29071 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:55.257050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:50791 (local address 127.19.113.126:36125)
0619 02:45:55.257123 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:55.257125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:55.257134 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:55.257181 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:55.257183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:55.257196 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:55.257245 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:55.257560 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.258204 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.258856 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.259055 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.259110 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:55.261398 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:50791: BlockingRecv error: recv got EOF from 127.19.113.66:50791 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 02:45:55 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:55 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:55.378551 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:55.374394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:42749)
0619 02:45:55.374525 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:55.374536 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:55.374596 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:55.374693 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:55.374694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:55.374848 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:55.374929 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:55.374931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.375542 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.375543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:55.376085 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:55.376087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.376447 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.376448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:55.376495 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:55.376922 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:55.376927 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:55.378491 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
W20260619 02:45:55.378635 29071 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:55.374441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:42749 (local address 127.19.113.126:36125)
0619 02:45:55.374488 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:55.374491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:55.374592 (+ 101us) server_negotiation.cc:407] Connection header received
0619 02:45:55.374635 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:55.374635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:55.374647 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:55.374688 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:55.374979 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.375485 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.376185 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:55.376375 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:55.376431 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:55.378576 (+ 2145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:42749: BlockingRecv error: recv got EOF from 127.19.113.67:42749 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 02:45:56 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:56 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:56.124378 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:56.119572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:50475)
0619 02:45:56.119706 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:56.119716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:56.119811 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:56.119951 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:56.119952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:56.120130 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:56.120178 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:56.120180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.120880 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.120881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:56.121489 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:56.121492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.121823 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.121824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:56.121871 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:56.122345 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:56.122349 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:56.124319 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
W20260619 02:45:56.124503 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:56.119684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:50475 (local address 127.19.113.126:36125)
0619 02:45:56.119799 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:56.119801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:56.119812 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:56.119875 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:56.119876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:56.119888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:56.119938 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:56.120246 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.120777 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.121556 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.121754 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.121808 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:56.124433 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:50475: BlockingRecv error: recv got EOF from 127.19.113.65:50475 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":50,"threads_started":1}
Jun 19 02:45:56 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:56 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:56.273236 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:56.261901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:44931)
0619 02:45:56.262121 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:56.262134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:56.262199 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:56.262517 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:56.262518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:56.262700 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:56.262765 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:56.262767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.263536 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.263537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:56.264100 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:56.264102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.264508 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.264509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:56.264605 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:56.271333 (+ 6728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:56.271339 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:56.273168 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":131,"threads_started":1}
W20260619 02:45:56.273422 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:56.262090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:44931 (local address 127.19.113.126:36125)
0619 02:45:56.262205 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:56.262214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:56.262341 (+ 127us) server_negotiation.cc:407] Connection header received
0619 02:45:56.262369 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:56.262370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:56.262389 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:56.262440 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:56.262943 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.263456 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.264193 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.264410 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.264471 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:56.273353 (+ 8882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:44931: BlockingRecv error: recv got EOF from 127.19.113.66:44931 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 02:45:56 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:56 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:56.383457 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:56.379052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:56131)
0619 02:45:56.379257 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:56.379269 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:56.379336 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:56.379479 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:56.379480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:56.379633 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:56.379680 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:56.379682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.380277 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.380277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:56.380844 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:56.380845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.381175 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.381176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:56.381226 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:56.381671 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:56.381675 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:56.383394 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":81,"threads_started":1}
W20260619 02:45:56.383555 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:56.379166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:56131 (local address 127.19.113.126:36125)
0619 02:45:56.379221 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:56.379223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:56.379330 (+ 107us) server_negotiation.cc:407] Connection header received
0619 02:45:56.379401 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:56.379402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:56.379413 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:56.379459 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:56.379730 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.380210 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.380902 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:56.381080 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:56.381124 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:56.383495 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:56131: BlockingRecv error: recv got EOF from 127.19.113.67:56131 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 02:45:57 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:57 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:57.129542 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:57.124833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:60485)
0619 02:45:57.124947 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:57.124960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:57.125026 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:57.125150 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:57.125151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:57.125308 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:57.125357 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:57.125358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.125991 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.125992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:57.126533 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:57.126535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.126976 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.126977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:57.127032 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:57.127488 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:57.127493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:57.129471 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
W20260619 02:45:57.129637 29079 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:57.124898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:60485 (local address 127.19.113.126:36125)
0619 02:45:57.125024 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:57.125026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:57.125036 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:57.125074 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:57.125075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:57.125084 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:57.125138 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:57.125411 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.125932 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.126617 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.126828 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.126971 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:57.129568 (+ 2597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:60485: BlockingRecv error: recv got EOF from 127.19.113.65:60485 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
Jun 19 02:45:57 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:57 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:57.278055 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:57.273858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:54175)
0619 02:45:57.273978 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:57.273990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:57.274046 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:57.274153 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:57.274154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:57.274308 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:57.274354 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:57.274356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.275056 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.275057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:57.275585 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:57.275587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.275893 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.275893 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:57.275934 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:57.276359 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:57.276362 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:57.277997 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
W20260619 02:45:57.278175 29079 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:57.273959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:54175 (local address 127.19.113.126:36125)
0619 02:45:57.274047 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:57.274049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:57.274076 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:45:57.274099 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:57.274100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:57.274111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:57.274149 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:57.274446 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.274991 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.275652 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.275835 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.275879 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:57.278106 (+ 2227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:54175: BlockingRecv error: recv got EOF from 127.19.113.66:54175 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 02:45:57 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:57 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:57.388641 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:57.383940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:60425)
0619 02:45:57.384106 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:57.384124 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:57.384217 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:57.384341 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:57.384342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:57.384501 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:57.384550 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:57.384552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.385213 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.385214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:57.385868 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:57.385871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.386291 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.386292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:57.386344 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:57.386793 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:57.386798 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:57.388581 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
W20260619 02:45:57.388734 29079 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:57.384092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:60425 (local address 127.19.113.126:36125)
0619 02:45:57.384184 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:57.384185 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:57.384200 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:45:57.384263 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:57.384263 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:57.384280 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:57.384333 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:57.384598 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.385083 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.385989 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:57.386221 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:57.386283 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:57.388673 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:60425: BlockingRecv error: recv got EOF from 127.19.113.67:60425 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 19 02:45:58 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:58 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:58.134987 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:58.130046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:33789)
0619 02:45:58.130193 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:58.130203 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:58.130253 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:58.130456 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:58.130457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:58.130642 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:58.130699 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:58.130701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.131430 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.131430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:58.132027 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:58.132030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.132460 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.132461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:58.132516 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:58.132989 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:58.132993 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:58.134917 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
W20260619 02:45:58.135109 29083 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:58.130173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:33789 (local address 127.19.113.126:36125)
0619 02:45:58.130296 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:58.130298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:58.130305 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:58.130328 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:58.130329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:58.130339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:58.130404 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:58.130783 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.131352 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.132134 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.132365 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.132424 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:58.135037 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:33789: BlockingRecv error: recv got EOF from 127.19.113.65:33789 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
Jun 19 02:45:58 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:58 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:58.283175 29084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:58.278559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:55517)
0619 02:45:58.278755 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:58.278769 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:58.278852 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:58.279034 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:58.279035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:58.279215 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:58.279260 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:58.279262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.279918 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.279919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:58.280467 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:58.280469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.280807 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.280809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:58.280866 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:58.281335 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:58.281340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:58.283106 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":62,"threads_started":1}
W20260619 02:45:58.283262 29083 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:58.278712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:55517 (local address 127.19.113.126:36125)
0619 02:45:58.278774 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:58.278776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:58.278832 (+ 56us) server_negotiation.cc:407] Connection header received
0619 02:45:58.278952 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:58.278953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:58.278965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:58.279013 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:58.279347 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.279840 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.280553 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.280741 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.280797 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:58.283204 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:55517: BlockingRecv error: recv got EOF from 127.19.113.66:55517 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 02:45:58 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:58 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:58.393823 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:58.389143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:35043)
0619 02:45:58.389309 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:58.389319 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:58.389379 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:58.389510 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:58.389510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:58.389720 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:58.389767 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:58.389769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.390540 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.390541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:58.391140 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:58.391143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.391481 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.391481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:58.391518 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:58.392012 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:58.392017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:58.393734 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
W20260619 02:45:58.393882 29083 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:58.389233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:35043 (local address 127.19.113.126:36125)
0619 02:45:58.389321 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:58.389323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:58.389366 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:45:58.389423 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:58.389424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:58.389438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:58.389497 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:58.389836 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.390481 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.391225 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:58.391415 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:58.391477 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:58.393828 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:35043: BlockingRecv error: recv got EOF from 127.19.113.67:35043 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 19 02:45:59 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:59 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:59.140128 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:59.135473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:59513)
0619 02:45:59.135629 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:59.135639 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:59.135686 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:59.135818 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:59.135819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:59.135991 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:59.136035 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:59.136037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.136741 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.136742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:59.137284 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:59.137286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.137667 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.137668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:59.137717 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:59.138274 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:59.138279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:59.140067 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
W20260619 02:45:59.140237 29087 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:59.135550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:59513 (local address 127.19.113.126:36125)
0619 02:45:59.135689 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:59.135690 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:59.135700 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:59.135742 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:59.135742 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:59.135753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:59.135804 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:59.136109 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.136645 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.137381 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.137584 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.137645 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:59.140167 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:59513: BlockingRecv error: recv got EOF from 127.19.113.65:59513 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
Jun 19 02:45:59 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:59 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:59.288064 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:59.283643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:54273)
0619 02:45:59.283811 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:59.283824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:59.283875 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:59.283998 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:59.283999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:59.284153 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:59.284197 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:59.284198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.284822 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.284823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:59.285359 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:59.285361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.285744 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.285745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:59.285788 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:59.286229 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:59.286233 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:59.288007 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
W20260619 02:45:59.288163 29087 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:59.283777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:54273 (local address 127.19.113.126:36125)
0619 02:45:59.283854 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:59.283855 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:59.283865 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:45:59.283927 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:59.283927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:59.283938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:59.283983 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:59.284245 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.284728 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.285445 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.285674 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.285734 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:59.288104 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:54273: BlockingRecv error: recv got EOF from 127.19.113.66:54273 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 02:45:59 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:45:59 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:45:59.398552 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:59.394232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:38559)
0619 02:45:59.394351 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:59.394362 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:59.394429 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:59.394551 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:59.394552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:59.394713 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:59.394760 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:59.394761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.395439 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.395439 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:59.396004 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:59.396006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.396431 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.396432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:59.396473 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:59.396878 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:59.396882 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:59.398477 (+ 1595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
W20260619 02:45:59.398661 29087 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:59.394311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:38559 (local address 127.19.113.126:36125)
0619 02:45:59.394369 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:59.394371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:59.394410 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:45:59.394480 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:59.394481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:59.394493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:59.394541 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:59.394811 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.395352 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.396114 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:59.396359 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:59.396421 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:59.398599 (+ 2178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:38559: BlockingRecv error: recv got EOF from 127.19.113.67:38559 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 02:46:00 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:46:00 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:46:00.145783 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:00.140666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:40149)
0619 02:46:00.140841 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:00.140853 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:00.140909 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:00.141158 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:00.141160 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:00.141327 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:00.141374 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:00.141376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.142128 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.142129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:00.142727 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:00.142730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.143179 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.143180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:00.143248 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:00.143763 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:00.143768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:00.145707 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":64,"threads_started":1}
W20260619 02:46:00.145963 29091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:00.140873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:40149 (local address 127.19.113.126:36125)
0619 02:46:00.141003 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:00.141005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:00.141013 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:46:00.141036 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:00.141037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:00.141046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:00.141107 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:00.141484 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.142037 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.142808 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.143092 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.143148 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:00.145888 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:40149: BlockingRecv error: recv got EOF from 127.19.113.65:40149 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
Jun 19 02:46:00 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:46:00 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:46:00.293110 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:00.288577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:42541)
0619 02:46:00.288706 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:00.288716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:00.288773 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:00.288897 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:00.288898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:00.289051 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:00.289095 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:00.289097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.289737 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.289738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:00.290274 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:00.290275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.290656 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.290657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:00.290693 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:00.291142 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:00.291148 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:00.293010 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
W20260619 02:46:00.293201 29091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:00.288713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:42541 (local address 127.19.113.126:36125)
0619 02:46:00.288773 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:00.288775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:00.288801 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:46:00.288829 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:00.288829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:00.288840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:00.288880 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:00.289163 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.289678 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.290395 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.290590 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.290653 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:00.293116 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:42541: BlockingRecv error: recv got EOF from 127.19.113.66:42541 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 02:46:00 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:46:00 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:46:00.403690 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:00.399077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:39069)
0619 02:46:00.399202 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:00.399213 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:00.399274 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:00.399413 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:00.399414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:00.399573 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:00.399623 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:00.399624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.400342 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.400343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:00.400866 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:00.400867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.401168 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.401168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:00.401209 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:00.401740 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:00.401745 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:00.403606 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
W20260619 02:46:00.403748 29091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:00.399158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:39069 (local address 127.19.113.126:36125)
0619 02:46:00.399246 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:00.399248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:00.399261 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:46:00.399329 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:00.399329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:00.399343 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:00.399401 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:00.399717 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.400217 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.400933 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:00.401113 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:00.401156 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:00.403699 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:39069: BlockingRecv error: recv got EOF from 127.19.113.67:39069 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 02:46:01 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:46:01 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:46:01.150799 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:01.146267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:39389)
0619 02:46:01.146423 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:01.146434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:01.146483 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:01.146618 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:01.146619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:01.146789 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:01.146840 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:01.146842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.147544 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.147545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:01.148102 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:01.148104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.148468 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.148469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:01.148520 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:01.148930 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:01.148934 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:01.150738 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
W20260619 02:46:01.150954 29095 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:01.146349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:39389 (local address 127.19.113.126:36125)
0619 02:46:01.146492 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:01.146494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:01.146522 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:46:01.146544 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:01.146544 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:01.146554 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:01.146605 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:01.146946 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.147471 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.148179 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.148394 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.148445 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:01.150837 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:39389: BlockingRecv error: recv got EOF from 127.19.113.65:39389 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":57,"threads_started":1}
Jun 19 02:46:01 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:46:01 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:46:01.303386 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:01.293587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:51459)
0619 02:46:01.293699 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:01.293709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:01.293753 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:01.293884 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:01.293885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:01.294056 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:01.294103 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:01.294105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.294848 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.294849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:01.295509 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:01.295513 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.295890 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.295891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:01.295933 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:01.301291 (+ 5358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:01.301296 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:01.303306 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
W20260619 02:46:01.303488 29095 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:01.293659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:51459 (local address 127.19.113.126:36125)
0619 02:46:01.293726 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:01.293728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:01.293739 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:46:01.293812 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:01.293813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:01.293823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:01.293873 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:01.294192 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.294683 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.295607 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.295820 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.295882 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:01.303434 (+ 7552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:51459: BlockingRecv error: recv got EOF from 127.19.113.66:51459 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 02:46:01 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:46:01 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.67@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:46:01.408957 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:01.404248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.67:60709)
0619 02:46:01.404418 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:01.404428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:01.404490 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:01.404617 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:01.404617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:01.404816 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:01.404865 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:01.404867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.405562 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.405563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:01.406100 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:01.406107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.406550 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.406551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:01.406613 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:01.407161 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:01.407166 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:01.408899 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
W20260619 02:46:01.409053 29095 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:01.404346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:60709 (local address 127.19.113.126:36125)
0619 02:46:01.404403 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:01.404405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:01.404485 (+ 80us) server_negotiation.cc:407] Connection header received
0619 02:46:01.404535 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:01.404535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:01.404547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:01.404605 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:01.404933 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.405463 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.406202 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:01.406465 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:01.406535 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:01.408991 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.67:60709: BlockingRecv error: recv got EOF from 127.19.113.67:60709 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 02:46:02 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:46:02 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.65@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:46:02.156241 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.151390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.65:48247)
0619 02:46:02.151500 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:02.151512 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:02.151559 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:02.151808 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:02.151809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:02.151962 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:02.152010 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.152011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.152747 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.152748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.153312 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.153314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.153712 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.153713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.153779 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.154261 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:02.154267 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:02.156170 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
W20260619 02:46:02.156378 29099 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:02.151491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:48247 (local address 127.19.113.126:36125)
0619 02:46:02.151627 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:02.151629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:02.151637 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:46:02.151662 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:02.151662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:02.151672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:02.151788 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:02.152111 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.152662 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.153412 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.153617 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.153678 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.156282 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.65:48247: BlockingRecv error: recv got EOF from 127.19.113.65:48247 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":55,"threads_started":1}
Jun 19 02:46:02 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:46:02 dist-test-slave-zqpb krb5kdc[28041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.113.66@KRBTEST.COM for oryx/127.19.113.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:46:02.308887 29100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.304053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36125 (local address 127.19.113.66:46281)
0619 02:46:02.304186 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:02.304197 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:02.304283 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:02.304442 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:02.304443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:02.304600 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:02.304647 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.304649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.305346 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.305347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.305882 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.305884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.306238 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.306240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.306304 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.306779 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:02.306784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:02.308798 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.113.126:36125: Server oryx/127.19.113.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
W20260619 02:46:02.309000 29099 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:02.304173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:46281 (local address 127.19.113.126:36125)
0619 02:46:02.304257 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:02.304259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:02.304270 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:46:02.304363 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:02.304364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:02.304376 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:02.304420 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:02.304726 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.305262 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.305968 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.306163 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.306220 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.308922 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.113.66:46281: BlockingRecv error: recv got EOF from 127.19.113.66:46281 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:46:02.331024 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 28624
I20260619 02:46:02.335080 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 28776
I20260619 02:46:02.338837 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 28909
I20260619 02:46:02.342695 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 28554
2026-06-19T02:46:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17358 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T02:46:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:46:02Z Disabled control of system clock
I20260619 02:46:02.368563 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:37545
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:41747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:46:02.448698 29104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:02.448927 29104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:02.448946 29104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:02.450203 29104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:46:02.450248 29104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:02.450266 29104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:02.450277 29104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:46:02.450289 29104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:46:02.451727 29104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:41747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:37545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:37545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:02.451982 29104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:02.452306 29104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:02.457077 29112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:02.457072 29110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:02.457072 29109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:02.457316 29104 server_base.cc:1034] running on GCE node
I20260619 02:46:02.457538 29104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:02.457813 29104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:02.458925 29104 hybrid_clock.cc:648] HybridClock initialized: now 1781837162458908 us; error 27 us; skew 500 ppm
I20260619 02:46:02.460281 29104 webserver.cc:460] Webserver started at http://127.19.113.126:38767/ using document root <none> and password file <none>
I20260619 02:46:02.460495 29104 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:02.460543 29104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:02.460685 29104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:02.461746 29104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "2395075410912160b2b2ab5cb760c36e"
server_key_iv: "0c417be85a69b21036c1789433562149"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.462071 29104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "2395075410912160b2b2ab5cb760c36e"
server_key_iv: "0c417be85a69b21036c1789433562149"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.463403 29104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:46:02.464089 29118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.464285 29104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:46:02.464366 29104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "2395075410912160b2b2ab5cb760c36e"
server_key_iv: "0c417be85a69b21036c1789433562149"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.464432 29104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:02.477579 29104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:02.477892 29104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:02.478024 29104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:02.481951 29104 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:37545
I20260619 02:46:02.481989 29170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:37545 every 8 connection(s)
I20260619 02:46:02.482311 29104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:46:02.482764 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 29104
I20260619 02:46:02.482900 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:46:02.483104 19909 external_mini_cluster.cc:1428] Setting key 09bf2d7e3abb0b4a989881769d4ae944
I20260619 02:46:02.483132 29171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:02.486619 29171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb: Bootstrap starting.
I20260619 02:46:02.487257 29171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:02.487531 29171 log.cc:824] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:02.488157 29171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb: No bootstrap required, opened a new log
I20260619 02:46:02.488783 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.483657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.19.113.126:37545)
0619 02:46:02.483862 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:02.483869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:02.483893 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:46:02.484486 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:02.484492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:02.484552 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:02.484721 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:02.484896 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.485755 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.486831 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.487340 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.487477 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.488148 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:02.488152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:02.488153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:46:02.488348 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:02.488385 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:46:02.488396 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:02.488450 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:02.488531 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:46:02.488611 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260619 02:46:02.490070 29171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 37545 } }
I20260619 02:46:02.490203 29171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:02.490235 29171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12f040dbaf3e4e7dabc3b41d5b9146eb, State: Initialized, Role: FOLLOWER
I20260619 02:46:02.490342 29171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 37545 } }
I20260619 02:46:02.490411 29171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:46:02.490440 29171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:46:02.490486 29171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:02.491052 29171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 37545 } }
I20260619 02:46:02.491147 29171 leader_election.cc:302] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12f040dbaf3e4e7dabc3b41d5b9146eb; no voters:
I20260619 02:46:02.491288 29171 leader_election.cc:288] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:46:02.491346 29176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:46:02.491554 29171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:46:02.491539 29176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [term 1 LEADER]: Becoming Leader. State: Replica: 12f040dbaf3e4e7dabc3b41d5b9146eb, State: Running, Role: LEADER
I20260619 02:46:02.491689 29176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 37545 } }
I20260619 02:46:02.492116 29177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 37545 } } }
I20260619 02:46:02.492110 29178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12f040dbaf3e4e7dabc3b41d5b9146eb. Latest consensus state: current_term: 1 leader_uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f040dbaf3e4e7dabc3b41d5b9146eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 37545 } } }
I20260619 02:46:02.492280 29178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [sys.catalog]: This master's current role is: LEADER
I20260619 02:46:02.492269 29177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb [sys.catalog]: This master's current role is: LEADER
I20260619 02:46:02.492518 29185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:46:02.492985 29185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:46:02.499274 29185 catalog_manager.cc:1347] Generated new cluster ID: c5cf46cf05904f2ab73cc17a4fb6f2d0
I20260619 02:46:02.499339 29185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:46:02.508981 29185 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:46:02.509482 29185 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:46:02.514308 29185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12f040dbaf3e4e7dabc3b41d5b9146eb: Generated new TSK 0
I20260619 02:46:02.514488 29185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:46:02.524884 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:37545
--builtin_ntp_servers=127.19.113.84:41747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 02:46:02.604890 29195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:02.605096 29195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:02.605125 29195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:02.606335 29195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:02.606376 29195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:02.606411 29195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:46:02.607795 29195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:41747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:02.608016 29195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:02.608292 29195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:02.613116 29200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:02.613183 29201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:02.613315 29195 server_base.cc:1034] running on GCE node
W20260619 02:46:02.613116 29203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:02.613651 29195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:02.613888 29195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:02.615075 29195 hybrid_clock.cc:648] HybridClock initialized: now 1781837162615055 us; error 32 us; skew 500 ppm
I20260619 02:46:02.616333 29195 webserver.cc:460] Webserver started at http://127.19.113.65:46739/ using document root <none> and password file <none>
I20260619 02:46:02.616523 29195 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:02.616564 29195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:02.616657 29195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:02.617739 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "f7fccc3ab25b4759af8e22c578ba63ea"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "0b6a0b017167cabe88217857737dbc3c"
server_key_iv: "542ed517ff7de1cdc48a89e5012b864b"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.618041 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "f7fccc3ab25b4759af8e22c578ba63ea"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "0b6a0b017167cabe88217857737dbc3c"
server_key_iv: "542ed517ff7de1cdc48a89e5012b864b"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.619388 29195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:46:02.620231 29209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.620445 29195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:46:02.620532 29195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "f7fccc3ab25b4759af8e22c578ba63ea"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "0b6a0b017167cabe88217857737dbc3c"
server_key_iv: "542ed517ff7de1cdc48a89e5012b864b"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.620591 29195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:02.631058 29195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:02.631356 29195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:02.631458 29195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:02.631654 29195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:02.631948 29195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:02.631978 29195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.632007 29195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:02.632022 29195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.637274 29195 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:37099
I20260619 02:46:02.637364 29322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:37099 every 8 connection(s)
I20260619 02:46:02.637643 29195 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:46:02.639873 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 29195
I20260619 02:46:02.639961 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:46:02.640144 19909 external_mini_cluster.cc:1428] Setting key 2140212b5b4de094a20b527d59579616
I20260619 02:46:02.641156 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:37545
--builtin_ntp_servers=127.19.113.84:41747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 02:46:02.643734 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.638395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:60353 (local address 127.19.113.126:37545)
0619 02:46:02.638486 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:02.638488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:02.638663 (+ 175us) server_negotiation.cc:407] Connection header received
0619 02:46:02.639230 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:02.639231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:02.639247 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:02.639306 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:02.640203 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.640857 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.641949 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.642237 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.642312 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.643420 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:02.643422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:02.643422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:46:02.643427 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:02.643442 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:46:02.643445 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:02.643492 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:02.643617 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 02:46:02.643655 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:46:02.643945 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.638362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:37545 (local address 127.19.113.65:60353)
0619 02:46:02.638549 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:02.638583 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:02.639109 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:02.639448 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:02.639451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:02.639460 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:02.640082 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.640097 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.640949 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.640949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.641853 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.641857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.642341 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.642342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.642504 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.643248 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:46:02.643249 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:02.643296 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:46:02.643324 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:02.643507 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:02.643511 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:02.643514 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:46:02.643579 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:46:02.643720 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260619 02:46:02.644519 29323 heartbeater.cc:347] Connected to a master server at 127.19.113.126:37545
I20260619 02:46:02.644618 29323 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:02.644804 29323 heartbeater.cc:511] Master 127.19.113.126:37545 requested a full tablet report, sending...
I20260619 02:46:02.645319 29135 ts_manager.cc:194] Registered new tserver with Master: f7fccc3ab25b4759af8e22c578ba63ea (127.19.113.65:37099)
I20260619 02:46:02.645933 29135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.65:60353
W20260619 02:46:02.729229 29326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:02.729446 29326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:02.729475 29326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:02.730666 29326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:02.730707 29326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:02.730741 29326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:46:02.732192 29326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:41747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:02.732432 29326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:02.732710 29326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:02.737491 29332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:02.737499 29334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:02.737499 29331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:02.737768 29326 server_base.cc:1034] running on GCE node
I20260619 02:46:02.737974 29326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:02.738216 29326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:02.739379 29326 hybrid_clock.cc:648] HybridClock initialized: now 1781837162739363 us; error 31 us; skew 500 ppm
I20260619 02:46:02.740712 29326 webserver.cc:460] Webserver started at http://127.19.113.66:33257/ using document root <none> and password file <none>
I20260619 02:46:02.740974 29326 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:02.741025 29326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:02.741133 29326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:02.742205 29326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "8d6b0c2001c7463f98d482de2206c0df"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "758c237b176c3bec153fe0a00f38c7fc"
server_key_iv: "87b9161d567d0c2d33d396a082ccca58"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.742534 29326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "8d6b0c2001c7463f98d482de2206c0df"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "758c237b176c3bec153fe0a00f38c7fc"
server_key_iv: "87b9161d567d0c2d33d396a082ccca58"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.743923 29326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:46:02.744721 29340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.744943 29326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:46:02.745028 29326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "8d6b0c2001c7463f98d482de2206c0df"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "758c237b176c3bec153fe0a00f38c7fc"
server_key_iv: "87b9161d567d0c2d33d396a082ccca58"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.745078 29326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:02.763679 29326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:02.763991 29326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:02.764117 29326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:02.764338 29326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:02.764662 29326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:02.764693 29326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.764722 29326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:02.764739 29326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.770635 29326 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:41779
I20260619 02:46:02.770675 29453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:41779 every 8 connection(s)
I20260619 02:46:02.771124 29326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:46:02.776320 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 29326
I20260619 02:46:02.776397 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.771860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:49967 (local address 127.19.113.126:37545)
0619 02:46:02.771916 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:02.771918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:02.772146 (+ 228us) server_negotiation.cc:407] Connection header received
0619 02:46:02.772673 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:02.772675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:02.772692 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:02.772753 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:02.773462 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.774012 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.775031 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.775222 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.775278 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.776096 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:02.776097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:02.776097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:46:02.776103 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:02.776118 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:46:02.776122 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:02.776162 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:02.776281 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 02:46:02.776311 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:46:02.776420 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:46:02.776597 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.771813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:37545 (local address 127.19.113.66:49967)
0619 02:46:02.772027 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:02.772061 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:02.772564 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:02.772833 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:02.772836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:02.772845 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:02.773366 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.773370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.774113 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.774114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.774938 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.774941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.775302 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.775304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.775448 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.775955 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:46:02.775956 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:02.775984 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:46:02.776008 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:02.776172 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:02.776177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:02.776181 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:46:02.776271 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:46:02.776383 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260619 02:46:02.776613 19909 external_mini_cluster.cc:1428] Setting key 5fa609513d4611c63f15ca8a2512edd6
I20260619 02:46:02.777166 29454 heartbeater.cc:347] Connected to a master server at 127.19.113.126:37545
I20260619 02:46:02.777272 29454 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:02.777469 29454 heartbeater.cc:511] Master 127.19.113.126:37545 requested a full tablet report, sending...
I20260619 02:46:02.777634 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:37545
--builtin_ntp_servers=127.19.113.84:41747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 02:46:02.777872 29135 ts_manager.cc:194] Registered new tserver with Master: 8d6b0c2001c7463f98d482de2206c0df (127.19.113.66:41779)
I20260619 02:46:02.778228 29135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.66:49967
W20260619 02:46:02.859319 29457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:02.859531 29457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:02.859560 29457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:02.860847 29457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:02.860903 29457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:02.860936 29457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:46:02.862354 29457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:41747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:02.862607 29457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:02.862933 29457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:02.867677 29462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:02.867678 29463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:02.867861 29465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:02.868211 29457 server_base.cc:1034] running on GCE node
I20260619 02:46:02.868449 29457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:02.868739 29457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:02.869899 29457 hybrid_clock.cc:648] HybridClock initialized: now 1781837162869872 us; error 44 us; skew 500 ppm
I20260619 02:46:02.871273 29457 webserver.cc:460] Webserver started at http://127.19.113.67:36813/ using document root <none> and password file <none>
I20260619 02:46:02.871490 29457 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:02.871538 29457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:02.871663 29457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:02.872716 29457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "2c718a1cb8b045f39a67b8b349b900b5"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "7ba38a52e2062879029e019aad569faa"
server_key_iv: "7794878ce96368e863a5c43d0fb72812"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.873072 29457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "2c718a1cb8b045f39a67b8b349b900b5"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "7ba38a52e2062879029e019aad569faa"
server_key_iv: "7794878ce96368e863a5c43d0fb72812"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.874297 29457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:46:02.875109 29471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.875280 29457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:46:02.875375 29457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "2c718a1cb8b045f39a67b8b349b900b5"
format_stamp: "Formatted at 2026-06-19 02:46:02 on dist-test-slave-zqpb"
server_key: "7ba38a52e2062879029e019aad569faa"
server_key_iv: "7794878ce96368e863a5c43d0fb72812"
server_key_version: "encryptionkey@0"
I20260619 02:46:02.875435 29457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:02.890995 29457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:02.891322 29457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:02.891454 29457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:02.891700 29457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:02.892040 29457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:02.892072 29457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.892102 29457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:02.892123 29457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:02.898195 29457 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:43285
I20260619 02:46:02.898240 29584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:43285 every 8 connection(s)
I20260619 02:46:02.898555 29457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:46:02.902540 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 29457
I20260619 02:46:02.902644 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:46:02.902846 19909 external_mini_cluster.cc:1428] Setting key 5189a078c82c025328b42bb0877cb580
I20260619 02:46:02.904126 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.899320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:53887 (local address 127.19.113.126:37545)
0619 02:46:02.899367 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:02.899369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:02.899633 (+ 264us) server_negotiation.cc:407] Connection header received
0619 02:46:02.900089 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:02.900091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:02.900101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:02.900144 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:02.900836 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.901362 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.902464 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.902746 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.902929 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.903870 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:02.903872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:02.903872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:46:02.903877 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:02.903892 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:46:02.903897 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:02.903942 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:02.904018 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:46:02.904046 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:46:02.904333 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:02.899325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:37545 (local address 127.19.113.67:53887)
0619 02:46:02.899478 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:02.899512 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:02.900000 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:02.900206 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:02.900209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:02.900218 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:02.900707 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.900711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.901463 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.901464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.902363 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:02.902366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:02.902925 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:02.902926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:02.903109 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:02.903671 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:46:02.903671 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:02.903701 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:46:02.903725 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:02.903951 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:02.903957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:02.903960 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:46:02.904022 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:46:02.904129 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260619 02:46:02.904814 29585 heartbeater.cc:347] Connected to a master server at 127.19.113.126:37545
I20260619 02:46:02.904901 29585 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:02.905087 29585 heartbeater.cc:511] Master 127.19.113.126:37545 requested a full tablet report, sending...
I20260619 02:46:02.905469 29135 ts_manager.cc:194] Registered new tserver with Master: 2c718a1cb8b045f39a67b8b349b900b5 (127.19.113.67:43285)
I20260619 02:46:02.905826 29135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.67:53887
I20260619 02:46:02.914773 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:46:02.916798 29174 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:02.916269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.19.113.126:37545)
0619 02:46:02.916331 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:02.916334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:02.916439 (+ 105us) server_negotiation.cc:407] Connection header received
0619 02:46:02.916511 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:02.916512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:02.916522 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:02.916568 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:02.916718 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45804: BlockingRecv error: recv got EOF from 127.0.0.1:45804 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:46:02.917526 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 29195
I20260619 02:46:02.921550 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 29326
I20260619 02:46:02.924973 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 29457
I20260619 02:46:02.928642 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 29104
2026-06-19T02:46:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (585 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T02:46:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:46:02Z Disabled control of system clock
I20260619 02:46:02.954128 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46153
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:46153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:46:03.036437 29596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:03.036654 29596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:03.036674 29596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:03.037853 29596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:46:03.037890 29596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:03.037900 29596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:03.037911 29596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:46:03.037922 29596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:46:03.039315 29596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:46153
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:03.039526 29596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:03.039791 29596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:03.044531 29602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:03.044541 29604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:03.044633 29596 server_base.cc:1034] running on GCE node
W20260619 02:46:03.044554 29601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:03.045007 29596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:03.045276 29596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:03.046416 29596 hybrid_clock.cc:648] HybridClock initialized: now 1781837163046403 us; error 31 us; skew 500 ppm
I20260619 02:46:03.047627 29596 webserver.cc:460] Webserver started at http://127.19.113.126:37825/ using document root <none> and password file <none>
I20260619 02:46:03.047857 29596 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:03.047897 29596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:03.047989 29596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:03.049021 29596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "56a3bb5530ab43db8a53a91473f7ce76"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "a4d91f6925aa9762c845c6852410436f"
server_key_iv: "fba5f1e981e0f76f4fbeaae3cbb9d458"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.049357 29596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "56a3bb5530ab43db8a53a91473f7ce76"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "a4d91f6925aa9762c845c6852410436f"
server_key_iv: "fba5f1e981e0f76f4fbeaae3cbb9d458"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.050549 29596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:46:03.051328 29610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.051504 29596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:46:03.051586 29596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "56a3bb5530ab43db8a53a91473f7ce76"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "a4d91f6925aa9762c845c6852410436f"
server_key_iv: "fba5f1e981e0f76f4fbeaae3cbb9d458"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.051651 29596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:03.055043 29596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:03.055325 29596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:03.055445 29596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:03.059141 29596 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:46153
I20260619 02:46:03.059172 29662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:46153 every 8 connection(s)
I20260619 02:46:03.059489 29596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:46:03.060125 29663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:03.062819 29663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76: Bootstrap starting.
I20260619 02:46:03.063447 29663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:03.063724 29663 log.cc:824] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:03.064450 29663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76: No bootstrap required, opened a new log
I20260619 02:46:03.066125 29663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a3bb5530ab43db8a53a91473f7ce76" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46153 } }
I20260619 02:46:03.066273 29663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:03.066294 29663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a3bb5530ab43db8a53a91473f7ce76, State: Initialized, Role: FOLLOWER
I20260619 02:46:03.066429 29663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a3bb5530ab43db8a53a91473f7ce76" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46153 } }
I20260619 02:46:03.066502 29663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:46:03.066529 29663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:46:03.066567 29663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:03.067255 29663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a3bb5530ab43db8a53a91473f7ce76" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46153 } }
I20260619 02:46:03.067346 29663 leader_election.cc:302] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56a3bb5530ab43db8a53a91473f7ce76; no voters:
I20260619 02:46:03.067528 29663 leader_election.cc:288] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:46:03.067607 29666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:46:03.067768 29666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [term 1 LEADER]: Becoming Leader. State: Replica: 56a3bb5530ab43db8a53a91473f7ce76, State: Running, Role: LEADER
I20260619 02:46:03.067770 29663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:46:03.067876 29666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a3bb5530ab43db8a53a91473f7ce76" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46153 } }
I20260619 02:46:03.068255 29667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56a3bb5530ab43db8a53a91473f7ce76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a3bb5530ab43db8a53a91473f7ce76" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46153 } } }
I20260619 02:46:03.068336 29667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [sys.catalog]: This master's current role is: LEADER
I20260619 02:46:03.068603 29668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56a3bb5530ab43db8a53a91473f7ce76. Latest consensus state: current_term: 1 leader_uuid: "56a3bb5530ab43db8a53a91473f7ce76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a3bb5530ab43db8a53a91473f7ce76" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46153 } } }
I20260619 02:46:03.068678 29668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76 [sys.catalog]: This master's current role is: LEADER
I20260619 02:46:03.068905 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 29596
I20260619 02:46:03.069011 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:46:03.069224 19909 external_mini_cluster.cc:1428] Setting key 8ef335430f80bd48e26fecaf0e3a6945
W20260619 02:46:03.069592 29682 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:46:03.069655 29682 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:46:03.069708 29675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:46:03.069875 29675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:46:03.071524 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:03.069727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.19.113.126:46153)
0619 02:46:03.070015 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:03.070023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:03.070048 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:46:03.070145 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:03.070151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:03.070195 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:03.070313 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:03.071079 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:03.071083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:03.071083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:46:03.071087 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:03.071134 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:46:03.071146 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:03.071205 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:03.071282 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:46:03.071342 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":70,"threads_started":1}
I20260619 02:46:03.071933 29675 catalog_manager.cc:1347] Generated new cluster ID: c71cf90000934d36b148a24b084cdbc0
I20260619 02:46:03.071993 29675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:46:03.098809 29675 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:46:03.099011 29675 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:46:03.101110 29675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56a3bb5530ab43db8a53a91473f7ce76: Generated new TSK 0
I20260619 02:46:03.101365 29675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:46:03.108902 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46153
--builtin_ntp_servers=127.19.113.84:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:46:03.192471 29687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:03.192741 29687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:03.192780 29687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:03.194775 29687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:03.194842 29687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:03.194916 29687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:46:03.197208 29687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:03.197691 29687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:03.198118 29687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:03.204200 29693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:03.204383 29692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:03.204483 29687 server_base.cc:1034] running on GCE node
W20260619 02:46:03.204774 29695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:03.205022 29687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:03.205281 29687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:03.206426 29687 hybrid_clock.cc:648] HybridClock initialized: now 1781837163206392 us; error 44 us; skew 500 ppm
I20260619 02:46:03.208036 29687 webserver.cc:460] Webserver started at http://127.19.113.65:38649/ using document root <none> and password file <none>
I20260619 02:46:03.208292 29687 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:03.208352 29687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:03.208468 29687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:03.209834 29687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "b731ac5e23454decad5230f588183532"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "c3396cd0bece22008c0fd7b6aebeb944"
server_key_iv: "1bb3994e13eb2c077f5e77bb6e1335f2"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.210237 29687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "b731ac5e23454decad5230f588183532"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "c3396cd0bece22008c0fd7b6aebeb944"
server_key_iv: "1bb3994e13eb2c077f5e77bb6e1335f2"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.211920 29687 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260619 02:46:03.213003 29701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.213162 29687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:46:03.213294 29687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "b731ac5e23454decad5230f588183532"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "c3396cd0bece22008c0fd7b6aebeb944"
server_key_iv: "1bb3994e13eb2c077f5e77bb6e1335f2"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.213374 29687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:03.217914 29687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:03.218253 29687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:03.218398 29687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:03.218636 29687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:03.219010 29687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:03.219055 29687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.219097 29687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:03.219123 29687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.226316 29687 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:33951
I20260619 02:46:03.226361 29814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:33951 every 8 connection(s)
I20260619 02:46:03.226742 29687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:46:03.230556 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:03.227989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:51703 (local address 127.19.113.126:46153)
0619 02:46:03.228060 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:03.228062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:03.228073 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:46:03.228795 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:03.228796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:03.228805 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:03.228836 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:03.230284 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:03.230286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:03.230286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:46:03.230291 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:03.230310 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:46:03.230314 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:03.230352 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:03.230466 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 02:46:03.230483 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:46:03.230993 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:03.227666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46153 (local address 127.19.113.65:51703)
0619 02:46:03.227889 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:03.227946 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:03.228659 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:03.229037 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:03.229042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:03.229053 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:03.230106 (+ 1053us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:46:03.230107 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:03.230146 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:46:03.230183 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:03.230390 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:03.230395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:03.230398 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:46:03.230694 (+ 296us) client_negotiation.cc:240] Negotiation successful
0619 02:46:03.230762 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260619 02:46:03.231487 29815 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46153
I20260619 02:46:03.231578 29815 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:03.231717 29815 heartbeater.cc:511] Master 127.19.113.126:46153 requested a full tablet report, sending...
I20260619 02:46:03.232216 29627 ts_manager.cc:194] Registered new tserver with Master: b731ac5e23454decad5230f588183532 (127.19.113.65:33951)
I20260619 02:46:03.233968 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 29687
I20260619 02:46:03.234056 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:46:03.234239 19909 external_mini_cluster.cc:1428] Setting key e91346fa94e4082aa625fd9c8494936e
I20260619 02:46:03.235278 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46153
--builtin_ntp_servers=127.19.113.84:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:46:03.314775 29818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:03.315034 29818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:03.315057 29818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:03.316191 29818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:03.316224 29818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:03.316248 29818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:46:03.317627 29818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:03.317847 29818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:03.318125 29818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:46:03.323191 29818 server_base.cc:1034] running on GCE node
W20260619 02:46:03.323242 29824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:03.323333 29823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:03.323395 29826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:03.323572 29818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:03.323864 29818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:03.325014 29818 hybrid_clock.cc:648] HybridClock initialized: now 1781837163325002 us; error 26 us; skew 500 ppm
I20260619 02:46:03.326234 29818 webserver.cc:460] Webserver started at http://127.19.113.66:45915/ using document root <none> and password file <none>
I20260619 02:46:03.326447 29818 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:03.326491 29818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:03.326606 29818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:03.327746 29818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "da9b7acd2f2a40db9d93f8f3a3079308"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "6b8e908888ba72b26d1a1577071d35c5"
server_key_iv: "2430c891437785e403236ed620b28e3f"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.328068 29818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "da9b7acd2f2a40db9d93f8f3a3079308"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "6b8e908888ba72b26d1a1577071d35c5"
server_key_iv: "2430c891437785e403236ed620b28e3f"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.329282 29818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:46:03.330104 29832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.330252 29818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:46:03.330358 29818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "da9b7acd2f2a40db9d93f8f3a3079308"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "6b8e908888ba72b26d1a1577071d35c5"
server_key_iv: "2430c891437785e403236ed620b28e3f"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.330417 29818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:03.333786 29818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:03.334031 29818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:03.334152 29818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:03.334345 29818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:03.334653 29818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:03.334689 29818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.334715 29818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:03.334740 29818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.340170 29818 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:42149
I20260619 02:46:03.340220 29945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:42149 every 8 connection(s)
I20260619 02:46:03.340523 29818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:46:03.343492 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:03.341216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:55249 (local address 127.19.113.126:46153)
0619 02:46:03.341284 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:03.341286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:03.341473 (+ 187us) server_negotiation.cc:407] Connection header received
0619 02:46:03.342004 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:03.342005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:03.342013 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:03.342059 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:03.343256 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:03.343258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:03.343258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:46:03.343262 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:03.343281 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:46:03.343285 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:03.343336 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:03.343405 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:46:03.343420 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:46:03.343665 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:03.341185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46153 (local address 127.19.113.66:55249)
0619 02:46:03.341362 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:03.341395 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:03.341882 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:03.342146 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:03.342150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:03.342160 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:03.343076 (+ 916us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:46:03.343076 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:03.343104 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:46:03.343170 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:03.343338 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:03.343342 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:03.343345 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:46:03.343412 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:46:03.343472 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260619 02:46:03.344099 29946 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46153
I20260619 02:46:03.344185 29946 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:03.344321 29946 heartbeater.cc:511] Master 127.19.113.126:46153 requested a full tablet report, sending...
I20260619 02:46:03.344748 29627 ts_manager.cc:194] Registered new tserver with Master: da9b7acd2f2a40db9d93f8f3a3079308 (127.19.113.66:42149)
I20260619 02:46:03.350219 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 29818
I20260619 02:46:03.350324 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:46:03.350482 19909 external_mini_cluster.cc:1428] Setting key 41a4baa2a290589847303f5d2d371fef
I20260619 02:46:03.351488 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46153
--builtin_ntp_servers=127.19.113.84:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:46:03.430646 29949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:03.430903 29949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:03.430936 29949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:03.432137 29949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:03.432179 29949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:03.432221 29949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:46:03.433677 29949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:03.433923 29949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:03.434217 29949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:03.439018 29957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:03.439008 29954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:03.439069 29949 server_base.cc:1034] running on GCE node
W20260619 02:46:03.439008 29955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:03.439394 29949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:03.439661 29949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:03.440846 29949 hybrid_clock.cc:648] HybridClock initialized: now 1781837163440832 us; error 27 us; skew 500 ppm
I20260619 02:46:03.442147 29949 webserver.cc:460] Webserver started at http://127.19.113.67:33733/ using document root <none> and password file <none>
I20260619 02:46:03.442374 29949 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:03.442425 29949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:03.442549 29949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:03.443665 29949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "818a89007af34509a8334a2751192da3"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "8f11cd31f5e628ae1476d28f9ee703de"
server_key_iv: "0fae72192c257bdf940670040e1bf5ba"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.444010 29949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "818a89007af34509a8334a2751192da3"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "8f11cd31f5e628ae1476d28f9ee703de"
server_key_iv: "0fae72192c257bdf940670040e1bf5ba"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.445253 29949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:46:03.445998 29963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.446167 29949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:46:03.446247 29949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "818a89007af34509a8334a2751192da3"
format_stamp: "Formatted at 2026-06-19 02:46:03 on dist-test-slave-zqpb"
server_key: "8f11cd31f5e628ae1476d28f9ee703de"
server_key_iv: "0fae72192c257bdf940670040e1bf5ba"
server_key_version: "encryptionkey@0"
I20260619 02:46:03.446310 29949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:03.449549 29949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:03.449817 29949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:03.449929 29949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:03.450099 29949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:03.450383 29949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:03.450415 29949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.450439 29949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:03.450464 29949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:03.455948 29949 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:34945
I20260619 02:46:03.456008 30076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:34945 every 8 connection(s)
I20260619 02:46:03.456301 29949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:46:03.459069 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:03.456978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:46003 (local address 127.19.113.126:46153)
0619 02:46:03.457032 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:03.457034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:03.457304 (+ 270us) server_negotiation.cc:407] Connection header received
0619 02:46:03.457705 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:03.457706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:03.457711 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:03.457770 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:03.458833 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:03.458834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:03.458834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:46:03.458837 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:03.458849 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:46:03.458852 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:03.458935 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:03.459003 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:46:03.459015 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:46:03.459259 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:03.456959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46153 (local address 127.19.113.67:46003)
0619 02:46:03.457130 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:03.457179 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:03.457627 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:03.457828 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:03.457831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:03.457839 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:03.458700 (+ 861us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:46:03.458701 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:03.458729 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:46:03.458757 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:03.458938 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:03.458942 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:03.458944 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:46:03.458998 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:46:03.459047 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260619 02:46:03.459717 30077 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46153
I20260619 02:46:03.459800 30077 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:03.459913 30077 heartbeater.cc:511] Master 127.19.113.126:46153 requested a full tablet report, sending...
I20260619 02:46:03.460203 29627 ts_manager.cc:194] Registered new tserver with Master: 818a89007af34509a8334a2751192da3 (127.19.113.67:34945)
I20260619 02:46:03.466168 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 29949
I20260619 02:46:03.466265 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:46:03.466415 19909 external_mini_cluster.cc:1428] Setting key a53be71bdfcc02843e5cf8a5b4cd29f4
I20260619 02:46:03.467243 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:46:03.469033 29685 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:46:03.468526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42246 (local address 127.19.113.126:46153)
0619 02:46:03.468606 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:03.468608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:03.468709 (+ 101us) server_negotiation.cc:407] Connection header received
0619 02:46:03.468785 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:03.468785 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:03.468790 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:03.468835 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:03.468968 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42246: BlockingRecv error: recv got EOF from 127.0.0.1:42246 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:46:03.469105 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 29687
I20260619 02:46:03.474287 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 29818
I20260619 02:46:03.478387 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 29949
I20260619 02:46:03.482378 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 29596
2026-06-19T02:46:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (553 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:46:03 dist-test-slave-zqpb krb5kdc[30087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:46:03 dist-test-slave-zqpb krb5kdc[30087](info): set up 2 sockets
Jun 19 02:46:03 dist-test-slave-zqpb krb5kdc[30087](info): commencing operation
krb5kdc: starting...
W20260619 02:46:04.439652 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:46:04 dist-test-slave-zqpb krb5kdc[30087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:46:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:46:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:46:04.557147 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36619
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:36619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:46:04.638949 30100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:04.639168 30100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:04.639190 30100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:04.640362 30100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:46:04.640400 30100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:04.640412 30100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:04.640424 30100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:46:04.640434 30100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:46:04.642163 30100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:36619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:04.642377 30100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:04.642565 30100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:04.647130 30108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:04.647127 30105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:04.647339 30106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:04.647382 30100 server_base.cc:1034] running on GCE node
I20260619 02:46:04.647583 30100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:04.647861 30100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:04.649035 30100 hybrid_clock.cc:648] HybridClock initialized: now 1781837164649021 us; error 30 us; skew 500 ppm
Jun 19 02:46:04 dist-test-slave-zqpb krb5kdc[30087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:46:04.651585 30100 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:46:04.652055 30100 webserver.cc:460] Webserver started at http://127.19.113.126:45719/ using document root <none> and password file <none>
I20260619 02:46:04.652274 30100 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:04.652333 30100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:04.652458 30100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:04.653524 30100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "c693ec6fba474a9ca68d72b16f8475ac"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "3bcf3be6f44b6097beee9066e9fc9b9c"
server_key_iv: "7a6ce801f0fa3417be7d68816462d195"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.653867 30100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "c693ec6fba474a9ca68d72b16f8475ac"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "3bcf3be6f44b6097beee9066e9fc9b9c"
server_key_iv: "7a6ce801f0fa3417be7d68816462d195"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.655180 30100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:46:04.655925 30115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:04.656078 30100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:46:04.656157 30100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "c693ec6fba474a9ca68d72b16f8475ac"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "3bcf3be6f44b6097beee9066e9fc9b9c"
server_key_iv: "7a6ce801f0fa3417be7d68816462d195"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.656225 30100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:04.670221 30100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:04.672093 30100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:04.672221 30100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:04.676196 30100 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:36619
I20260619 02:46:04.676235 30167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:36619 every 8 connection(s)
I20260619 02:46:04.676568 30100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:46:04.677498 30168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:04.679812 30168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac: Bootstrap starting.
I20260619 02:46:04.680392 30168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:04.680650 30168 log.cc:824] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:04.681401 30168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac: No bootstrap required, opened a new log
I20260619 02:46:04.681968 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 30100
I20260619 02:46:04.682060 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:46:04.682224 19909 external_mini_cluster.cc:1428] Setting key 11e511ccde614abd94c4ba4cc3d6b1b6
I20260619 02:46:04.683482 30168 raft_consensus.cc:348] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c693ec6fba474a9ca68d72b16f8475ac" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36619 } }
I20260619 02:46:04.683669 30168 raft_consensus.cc:374] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:04.683707 30168 raft_consensus.cc:729] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c693ec6fba474a9ca68d72b16f8475ac, State: Initialized, Role: FOLLOWER
I20260619 02:46:04.683840 30168 consensus_queue.cc:260] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c693ec6fba474a9ca68d72b16f8475ac" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36619 } }
I20260619 02:46:04.683959 30168 raft_consensus.cc:388] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:46:04.684005 30168 raft_consensus.cc:482] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:46:04.684056 30168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:04.685163 30168 raft_consensus.cc:504] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c693ec6fba474a9ca68d72b16f8475ac" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36619 } }
I20260619 02:46:04.685273 30168 leader_election.cc:302] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c693ec6fba474a9ca68d72b16f8475ac; no voters:
I20260619 02:46:04.685431 30168 leader_election.cc:288] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:46:04.685475 30173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:46:04.685631 30173 raft_consensus.cc:686] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [term 1 LEADER]: Becoming Leader. State: Replica: c693ec6fba474a9ca68d72b16f8475ac, State: Running, Role: LEADER
I20260619 02:46:04.685681 30168 sys_catalog.cc:564] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:46:04.685793 30173 consensus_queue.cc:237] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c693ec6fba474a9ca68d72b16f8475ac" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36619 } }
I20260619 02:46:04.686132 30175 sys_catalog.cc:455] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader c693ec6fba474a9ca68d72b16f8475ac. Latest consensus state: current_term: 1 leader_uuid: "c693ec6fba474a9ca68d72b16f8475ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c693ec6fba474a9ca68d72b16f8475ac" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36619 } } }
I20260619 02:46:04.686242 30175 sys_catalog.cc:458] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [sys.catalog]: This master's current role is: LEADER
I20260619 02:46:04.686132 30174 sys_catalog.cc:455] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c693ec6fba474a9ca68d72b16f8475ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c693ec6fba474a9ca68d72b16f8475ac" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36619 } } }
I20260619 02:46:04.686349 30174 sys_catalog.cc:458] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac [sys.catalog]: This master's current role is: LEADER
I20260619 02:46:04.686440 30182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:46:04.686951 30182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:46:04.688635 30182 catalog_manager.cc:1347] Generated new cluster ID: c1b8915258d1427bbbdaa710f1712978
I20260619 02:46:04.688684 30182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:46:04 dist-test-slave-zqpb krb5kdc[30087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:46:04.694664 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:04.682741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.19.113.126:36619)
0619 02:46:04.682996 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:04.683006 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:04.683045 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:46:04.683454 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:04.683462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:04.683528 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:04.683671 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:04.684371 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:04.685764 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:04.686906 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:04.687483 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:04.687909 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:04.690266 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:04.690271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:04.690271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:04.690283 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:04.692397 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:04.692843 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:04.692847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:04.692847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:04.692987 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:04.693222 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:04.693223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:04.693223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:04.693594 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:04.694214 (+ 620us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:04.694291 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:46:04.694443 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
I20260619 02:46:04.700161 30182 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:46:04.700908 30182 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:46:04.711547 30182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac: Generated new TSK 0
I20260619 02:46:04.711771 30182 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:46:04.739898 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36619
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:46:04.820711 30194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:04.820919 30194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:04.820942 30194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:04.822086 30194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:04.822127 30194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:04.822158 30194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:46:04.823638 30194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:04.823861 30194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:04.824077 30194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:04.828934 30199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:04.828934 30202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:04.828954 30200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:04.829350 30194 server_base.cc:1034] running on GCE node
I20260619 02:46:04.829561 30194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:04.829802 30194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:04.830924 30194 hybrid_clock.cc:648] HybridClock initialized: now 1781837164830906 us; error 29 us; skew 500 ppm
Jun 19 02:46:04 dist-test-slave-zqpb krb5kdc[30087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:46:04.833351 30194 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:46:04.833791 30194 webserver.cc:460] Webserver started at http://127.19.113.65:40735/ using document root <none> and password file <none>
I20260619 02:46:04.833973 30194 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:04.834012 30194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:04.834101 30194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:04.835150 30194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "5d3f6ec54abf4617b0de37662fc02dac"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "f8e774bee34b455ea10a82d3e7c302d3"
server_key_iv: "902b9fdb3d2c9da0d64b567c940f8dce"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.835445 30194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "5d3f6ec54abf4617b0de37662fc02dac"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "f8e774bee34b455ea10a82d3e7c302d3"
server_key_iv: "902b9fdb3d2c9da0d64b567c940f8dce"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.836643 30194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:46:04.837371 30209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:04.837561 30194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:46:04.837652 30194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "5d3f6ec54abf4617b0de37662fc02dac"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "f8e774bee34b455ea10a82d3e7c302d3"
server_key_iv: "902b9fdb3d2c9da0d64b567c940f8dce"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.837774 30194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:04.850155 30194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:04.852093 30194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:04.852210 30194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:04.852432 30194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:04.852756 30194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:04.852789 30194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:04.852811 30194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:04.852834 30194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:04.858333 30194 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:33561
I20260619 02:46:04.858398 30322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:33561 every 8 connection(s)
I20260619 02:46:04.858722 30194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:46:04.864537 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 30194
I20260619 02:46:04.864660 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:46:04.864912 19909 external_mini_cluster.cc:1428] Setting key d2cd5e94c9616f748b20a8f9cde928f9
Jun 19 02:46:04 dist-test-slave-zqpb krb5kdc[30087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:46:04.868410 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:04.859457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:33723 (local address 127.19.113.126:36619)
0619 02:46:04.859517 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:04.859519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:04.859685 (+ 166us) server_negotiation.cc:407] Connection header received
0619 02:46:04.860298 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:04.860299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:04.860312 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:04.860368 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:04.861300 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:04.861856 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:04.862842 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:04.863125 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:04.863182 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:04.865526 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:04.865527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:04.865528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:04.865535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:04.867016 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:04.867455 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:04.867456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:04.867457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:04.867520 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:04.867755 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:04.867756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:04.867756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:04.867879 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:04.867939 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:04.868231 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 02:46:04.868319 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:46:04.868571 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:04.859410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36619 (local address 127.19.113.65:33723)
0619 02:46:04.859574 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:04.859611 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:04.860187 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:04.860424 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:04.860428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:04.861005 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:04.861208 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:04.861212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:04.861940 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:04.861940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:04.862764 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:04.862766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:04.863220 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:04.863221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:04.863336 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:04.863830 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:04.863839 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:04.865426 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:04.867094 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:04.867101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:04.867103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:04.867366 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:04.867575 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:04.867580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:04.867581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:04.867681 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:04.867936 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:04.867944 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:04.868081 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 02:46:04.868195 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:46:04.868354 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260619 02:46:04.869298 30323 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36619
I20260619 02:46:04.869422 30323 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:04.869653 30323 heartbeater.cc:511] Master 127.19.113.126:36619 requested a full tablet report, sending...
I20260619 02:46:04.870254 30132 ts_manager.cc:194] Registered new tserver with Master: 5d3f6ec54abf4617b0de37662fc02dac (127.19.113.65:33561)
I20260619 02:46:04.870924 30132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:33723
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:46:04.890069 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36619
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:46:04.969288 30328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:04.969496 30328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:04.969527 30328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:04.970717 30328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:04.970760 30328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:04.970795 30328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:46:04.972317 30328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:04.972553 30328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:04.972779 30328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:04.977581 30336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:04.977587 30333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:04.977592 30334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:04.977869 30328 server_base.cc:1034] running on GCE node
I20260619 02:46:04.978026 30328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:04.978241 30328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:04.979379 30328 hybrid_clock.cc:648] HybridClock initialized: now 1781837164979362 us; error 27 us; skew 500 ppm
Jun 19 02:46:04 dist-test-slave-zqpb krb5kdc[30087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:46:04.981612 30328 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:46:04.982005 30328 webserver.cc:460] Webserver started at http://127.19.113.66:34671/ using document root <none> and password file <none>
I20260619 02:46:04.982198 30328 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:04.982244 30328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:04.982368 30328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:04.983418 30328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "9c2a0381a3314e83a52dfd7e89c2e606"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "baca829123dc3ef6a9b9aeadeb7d7d81"
server_key_iv: "fb0875baffdc2ea61c1cb76c8b71242b"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.983738 30328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "9c2a0381a3314e83a52dfd7e89c2e606"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "baca829123dc3ef6a9b9aeadeb7d7d81"
server_key_iv: "fb0875baffdc2ea61c1cb76c8b71242b"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.984983 30328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:46:04.985692 30343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:04.985872 30328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:46:04.985948 30328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "9c2a0381a3314e83a52dfd7e89c2e606"
format_stamp: "Formatted at 2026-06-19 02:46:04 on dist-test-slave-zqpb"
server_key: "baca829123dc3ef6a9b9aeadeb7d7d81"
server_key_iv: "fb0875baffdc2ea61c1cb76c8b71242b"
server_key_version: "encryptionkey@0"
I20260619 02:46:04.986054 30328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:04.999550 30328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:05.001513 30328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:05.001619 30328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:05.001786 30328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:05.002041 30328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:05.002070 30328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:05.002092 30328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:05.002139 30328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:05.007818 30328 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:43389
I20260619 02:46:05.007894 30456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:43389 every 8 connection(s)
I20260619 02:46:05.008168 30328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:46:05 dist-test-slave-zqpb krb5kdc[30087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:46:05.014619 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 30328
I20260619 02:46:05.014696 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:46:05.014917 19909 external_mini_cluster.cc:1428] Setting key 90e0a8bb09f614dc83938487c15757ab
I20260619 02:46:05.017396 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.008872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:57527 (local address 127.19.113.126:36619)
0619 02:46:05.008949 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.008951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.009119 (+ 168us) server_negotiation.cc:407] Connection header received
0619 02:46:05.009688 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.009689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.009700 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.009757 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:05.010678 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.011276 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.012242 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.012431 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.012482 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.014527 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:05.014528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:05.014529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:05.014535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:05.016150 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:05.016524 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:05.016525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:05.016525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:05.016590 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:05.016802 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:05.016803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:05.016803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:05.016915 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:05.016969 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.017215 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.017306 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:46:05.017566 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.008847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36619 (local address 127.19.113.66:57527)
0619 02:46:05.009008 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:05.009043 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:05.009509 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:05.009854 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:05.009858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:05.010388 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:05.010576 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.010580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.011341 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.011342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.012156 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.012158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.012489 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.012490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.012581 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.013061 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:05.013069 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:05.014434 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:05.016236 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:05.016243 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:05.016245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:05.016457 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:05.016643 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:05.016644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:05.016644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:05.016746 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:05.016966 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:05.016973 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:05.017099 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 02:46:05.017193 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 02:46:05.017352 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260619 02:46:05.018088 30457 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36619
I20260619 02:46:05.018181 30457 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:05.018348 30457 heartbeater.cc:511] Master 127.19.113.126:36619 requested a full tablet report, sending...
I20260619 02:46:05.018712 30132 ts_manager.cc:194] Registered new tserver with Master: 9c2a0381a3314e83a52dfd7e89c2e606 (127.19.113.66:43389)
I20260619 02:46:05.019246 30132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:57527
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:46:05.037312 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36619
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:46:05.115839 30462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:05.116050 30462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:05.116076 30462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:05.117183 30462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:05.117216 30462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:05.117240 30462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:46:05.118597 30462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:05.118785 30462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:05.118980 30462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:05.123625 30467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:05.123631 30470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:05.123631 30468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:05.123965 30462 server_base.cc:1034] running on GCE node
I20260619 02:46:05.124158 30462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:05.124387 30462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:05.125553 30462 hybrid_clock.cc:648] HybridClock initialized: now 1781837165125528 us; error 41 us; skew 500 ppm
Jun 19 02:46:05 dist-test-slave-zqpb krb5kdc[30087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837165, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:46:05.128015 30462 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:46:05.128463 30462 webserver.cc:460] Webserver started at http://127.19.113.67:41717/ using document root <none> and password file <none>
I20260619 02:46:05.128670 30462 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:05.128724 30462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:05.128846 30462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:46:05.129879 30462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "4552174bf13244208b017c94c0e833ed"
format_stamp: "Formatted at 2026-06-19 02:46:05 on dist-test-slave-zqpb"
server_key: "da7b46ea22e803fc0e8f1b36025277fa"
server_key_iv: "24c58194db88468fb18857d9edd7ecae"
server_key_version: "encryptionkey@0"
I20260619 02:46:05.130211 30462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "4552174bf13244208b017c94c0e833ed"
format_stamp: "Formatted at 2026-06-19 02:46:05 on dist-test-slave-zqpb"
server_key: "da7b46ea22e803fc0e8f1b36025277fa"
server_key_iv: "24c58194db88468fb18857d9edd7ecae"
server_key_version: "encryptionkey@0"
I20260619 02:46:05.131505 30462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:46:05.132231 30477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:05.132382 30462 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:46:05.132462 30462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "4552174bf13244208b017c94c0e833ed"
format_stamp: "Formatted at 2026-06-19 02:46:05 on dist-test-slave-zqpb"
server_key: "da7b46ea22e803fc0e8f1b36025277fa"
server_key_iv: "24c58194db88468fb18857d9edd7ecae"
server_key_version: "encryptionkey@0"
I20260619 02:46:05.132522 30462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:05.150750 30462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:05.152861 30462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:05.153007 30462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:05.153226 30462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:05.153520 30462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:05.153553 30462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:05.153580 30462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:05.153604 30462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:05.159052 30462 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:42767
I20260619 02:46:05.159108 30590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:42767 every 8 connection(s)
I20260619 02:46:05.159409 30462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:46:05.161850 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 30462
I20260619 02:46:05.161970 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:46:05.162153 19909 external_mini_cluster.cc:1428] Setting key f0516cc008c229d624a5311c28785dd0
Jun 19 02:46:05 dist-test-slave-zqpb krb5kdc[30087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837165, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:46:05.168239 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.160092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:35657 (local address 127.19.113.126:36619)
0619 02:46:05.160167 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.160169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.160354 (+ 185us) server_negotiation.cc:407] Connection header received
0619 02:46:05.160851 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.160852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.160864 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.160913 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:05.161908 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.162570 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.163658 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.163850 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.163911 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.165852 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:05.165853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:05.165853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:05.165859 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:05.167103 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:05.167464 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:05.167464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:05.167465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:05.167515 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:05.167720 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:05.167721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:05.167721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:05.167842 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:05.167901 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.168101 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.168166 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:46:05.168401 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.160090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36619 (local address 127.19.113.67:35657)
0619 02:46:05.160234 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:05.160269 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:05.160758 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:05.160984 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:05.160987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:05.161543 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:05.161800 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.161807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.162653 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.162654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.163569 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.163572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.163917 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.163918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.164012 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.164504 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:05.164512 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:05.165779 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:05.167169 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:05.167173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:05.167175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:05.167402 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:05.167564 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:05.167564 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:05.167564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:05.167649 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:05.167896 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:05.167899 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:05.168005 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 02:46:05.168081 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:46:05.168217 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260619 02:46:05.168793 30591 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36619
I20260619 02:46:05.168869 30591 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:05.169001 30591 heartbeater.cc:511] Master 127.19.113.126:36619 requested a full tablet report, sending...
I20260619 02:46:05.169320 30132 ts_manager.cc:194] Registered new tserver with Master: 4552174bf13244208b017c94c0e833ed (127.19.113.67:42767)
I20260619 02:46:05.169656 30132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:35657
I20260619 02:46:05.173753 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:46:05.180683 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.174952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.19.113.126:36619)
0619 02:46:05.175006 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.175008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.175111 (+ 103us) server_negotiation.cc:407] Connection header received
0619 02:46:05.175151 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.175152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.175159 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.175199 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:05.175683 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.176177 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.176932 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.177112 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.177156 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.178283 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:05.178284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:05.178284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:05.178290 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:05.179562 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:05.179913 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:05.179914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:05.179914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:05.179966 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:05.180221 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:05.180222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:05.180222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:05.180380 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:05.180460 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.180560 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.180618 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:46:05.182116 30132 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:46:05.183699 30132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:46:05.189517 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.185799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:42767 (local address 127.0.0.1:49368)
0619 02:46:05.186127 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:05.186137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:05.186222 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:05.186400 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:05.186403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:05.186405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:05.186508 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.186510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.187493 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.187494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.188496 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.188499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.189232 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.189232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.189310 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.189324 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:46:05.189393 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:46:05.189443 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":65,"threads_started":1}
I20260619 02:46:05.189532 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.185966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49368 (local address 127.19.113.67:42767)
0619 02:46:05.186133 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.186138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.186182 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:46:05.186299 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.186302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.186317 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.186382 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:05.186569 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.187405 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.188569 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.189161 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.189224 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.189332 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.189406 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.189442 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260619 02:46:05.190415 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.186183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.19.113.65:33561)
0619 02:46:05.186401 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.186405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.186453 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:46:05.186492 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.186494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.186509 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.186738 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:05.186811 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.187979 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.189367 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.190088 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.190153 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.190216 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.190303 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.190340 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":87,"threads_started":1}
I20260619 02:46:05.190405 30600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.185796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:43389 (local address 127.0.0.1:44008)
0619 02:46:05.185911 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:05.185950 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:05.186009 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:05.186698 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:05.186699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:05.186700 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:05.186747 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.186748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.188078 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.188080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.189249 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.189252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.190177 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.190178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.190254 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.190259 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:46:05.190287 (+ 28us) client_negotiation.cc:240] Negotiation successful
0619 02:46:05.190326 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":50,"thread_start_us":59,"threads_started":1}
I20260619 02:46:05.190408 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.185992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:33561 (local address 127.0.0.1:57962)
0619 02:46:05.186278 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:05.186287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:05.186332 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:05.186603 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:05.186604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:05.186607 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:05.186654 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.186655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.188098 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.188099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.189290 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.189292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.190156 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.190157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.190231 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.190236 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:46:05.190291 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 02:46:05.190333 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":1,"thread_start_us":61,"threads_started":1}
I20260619 02:46:05.190443 30525 tablet_service.cc:1467] Processing CreateTablet for tablet e587cd70b37145cbabcf1151c6423495 (DEFAULT_TABLE table=test-table [id=f6eb6b2418814fdd8872b7acf896f3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:46:05.190786 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.185884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.19.113.66:43389)
0619 02:46:05.186549 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.186553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.186562 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:46:05.186596 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.186598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.186611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.186812 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:05.186905 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.187963 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.189347 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.190080 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.190566 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.190633 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.190657 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.190698 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":71,"threads_started":1}
I20260619 02:46:05.190797 30525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e587cd70b37145cbabcf1151c6423495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:05.191169 30257 tablet_service.cc:1467] Processing CreateTablet for tablet e587cd70b37145cbabcf1151c6423495 (DEFAULT_TABLE table=test-table [id=f6eb6b2418814fdd8872b7acf896f3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:46:05.191378 30391 tablet_service.cc:1467] Processing CreateTablet for tablet e587cd70b37145cbabcf1151c6423495 (DEFAULT_TABLE table=test-table [id=f6eb6b2418814fdd8872b7acf896f3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:46:05.191478 30257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e587cd70b37145cbabcf1151c6423495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:05.191612 30391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e587cd70b37145cbabcf1151c6423495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:05.193997 30606 tablet_bootstrap.cc:492] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: Bootstrap starting.
I20260619 02:46:05.194161 30605 tablet_bootstrap.cc:492] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Bootstrap starting.
I20260619 02:46:05.194679 30606 tablet_bootstrap.cc:654] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:05.194691 30605 tablet_bootstrap.cc:654] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:05.194756 30607 tablet_bootstrap.cc:492] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac: Bootstrap starting.
I20260619 02:46:05.195286 30605 log.cc:824] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:05.195400 30606 log.cc:824] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:05.195567 30607 tablet_bootstrap.cc:654] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:05.196089 30605 tablet_bootstrap.cc:492] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: No bootstrap required, opened a new log
I20260619 02:46:05.196103 30607 log.cc:824] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:05.196148 30605 ts_tablet_manager.cc:1397] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:46:05.196195 30606 tablet_bootstrap.cc:492] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: No bootstrap required, opened a new log
I20260619 02:46:05.196259 30606 ts_tablet_manager.cc:1397] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:46:05.197618 30605 raft_consensus.cc:348] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.197662 30606 raft_consensus.cc:348] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.197758 30605 raft_consensus.cc:374] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:05.197779 30606 raft_consensus.cc:374] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:05.197790 30605 raft_consensus.cc:729] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a0381a3314e83a52dfd7e89c2e606, State: Initialized, Role: FOLLOWER
I20260619 02:46:05.197803 30606 raft_consensus.cc:729] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4552174bf13244208b017c94c0e833ed, State: Initialized, Role: FOLLOWER
I20260619 02:46:05.197902 30606 consensus_queue.cc:260] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.197903 30605 consensus_queue.cc:260] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.198146 30606 ts_tablet_manager.cc:1428] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:46:05.198150 30605 ts_tablet_manager.cc:1428] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:46:05.198169 30591 heartbeater.cc:503] Master 127.19.113.126:36619 was elected leader, sending a full tablet report...
I20260619 02:46:05.198437 30457 heartbeater.cc:503] Master 127.19.113.126:36619 was elected leader, sending a full tablet report...
I20260619 02:46:05.198803 30607 tablet_bootstrap.cc:492] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac: No bootstrap required, opened a new log
I20260619 02:46:05.198859 30607 ts_tablet_manager.cc:1397] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:46:05.200366 30607 raft_consensus.cc:348] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.200482 30607 raft_consensus.cc:374] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:05.200511 30607 raft_consensus.cc:729] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d3f6ec54abf4617b0de37662fc02dac, State: Initialized, Role: FOLLOWER
I20260619 02:46:05.200615 30607 consensus_queue.cc:260] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.200829 30607 ts_tablet_manager.cc:1428] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:46:05.200889 30323 heartbeater.cc:503] Master 127.19.113.126:36619 was elected leader, sending a full tablet report...
W20260619 02:46:05.258738 30458 tablet.cc:2369] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:46:05.326236 30613 raft_consensus.cc:482] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:46:05.326370 30613 raft_consensus.cc:504] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.326747 30613 leader_election.cc:288] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a0381a3314e83a52dfd7e89c2e606 (127.19.113.66:43389), 4552174bf13244208b017c94c0e833ed (127.19.113.67:42767)
I20260619 02:46:05.329411 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.326946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:42767 (local address 127.19.113.65:35255)
0619 02:46:05.326999 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:05.327012 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:05.327061 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:05.327178 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:05.327179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:05.327181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:05.327259 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.327261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.327893 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.327894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.328626 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.328628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.329169 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.329170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.329246 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.329250 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:46:05.329294 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:46:05.329335 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 02:46:05.329411 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.327033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:35255 (local address 127.19.113.67:42767)
0619 02:46:05.327076 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.327078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.327087 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:46:05.327113 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.327114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.327123 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.327171 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:05.327320 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.327812 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.328733 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.329104 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.329164 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.329174 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.329306 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.329343 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:46:05.329671 30614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.326956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:43389 (local address 127.19.113.65:47993)
0619 02:46:05.327160 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:05.327170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:05.327230 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:05.327347 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:05.327348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:05.327349 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:05.327416 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.327418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.327995 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.327995 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.328780 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:05.328783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.329296 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.329297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:05.329368 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.329371 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:46:05.329415 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:46:05.329450 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":114,"threads_started":1}
I20260619 02:46:05.329746 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.327076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:47993 (local address 127.19.113.66:43389)
0619 02:46:05.327128 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.327130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.327237 (+ 107us) server_negotiation.cc:407] Connection header received
0619 02:46:05.327285 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.327286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.327295 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.327340 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:05.327471 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.327925 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.328858 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.329227 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.329288 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.329296 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.329590 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.329648 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:46:05.329777 30545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e587cd70b37145cbabcf1151c6423495" candidate_uuid: "5d3f6ec54abf4617b0de37662fc02dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4552174bf13244208b017c94c0e833ed" is_pre_election: true
I20260619 02:46:05.329946 30545 raft_consensus.cc:2454] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d3f6ec54abf4617b0de37662fc02dac in term 0.
I20260619 02:46:05.330098 30411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e587cd70b37145cbabcf1151c6423495" candidate_uuid: "5d3f6ec54abf4617b0de37662fc02dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" is_pre_election: true
I20260619 02:46:05.330164 30213 leader_election.cc:302] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4552174bf13244208b017c94c0e833ed, 5d3f6ec54abf4617b0de37662fc02dac; no voters:
I20260619 02:46:05.330247 30411 raft_consensus.cc:2454] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d3f6ec54abf4617b0de37662fc02dac in term 0.
I20260619 02:46:05.330269 30613 raft_consensus.cc:2781] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:46:05.330327 30613 raft_consensus.cc:482] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:46:05.330435 30613 raft_consensus.cc:3037] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:05.331166 30613 raft_consensus.cc:504] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.331288 30613 leader_election.cc:288] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a0381a3314e83a52dfd7e89c2e606 (127.19.113.66:43389), 4552174bf13244208b017c94c0e833ed (127.19.113.67:42767)
I20260619 02:46:05.331455 30411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e587cd70b37145cbabcf1151c6423495" candidate_uuid: "5d3f6ec54abf4617b0de37662fc02dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a0381a3314e83a52dfd7e89c2e606"
I20260619 02:46:05.331507 30545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e587cd70b37145cbabcf1151c6423495" candidate_uuid: "5d3f6ec54abf4617b0de37662fc02dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4552174bf13244208b017c94c0e833ed"
I20260619 02:46:05.331526 30411 raft_consensus.cc:3037] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:05.331578 30545 raft_consensus.cc:3037] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:05.332208 30411 raft_consensus.cc:2454] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d3f6ec54abf4617b0de37662fc02dac in term 1.
I20260619 02:46:05.332257 30545 raft_consensus.cc:2454] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d3f6ec54abf4617b0de37662fc02dac in term 1.
I20260619 02:46:05.332379 30210 leader_election.cc:302] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d3f6ec54abf4617b0de37662fc02dac, 9c2a0381a3314e83a52dfd7e89c2e606; no voters:
I20260619 02:46:05.332475 30613 raft_consensus.cc:2781] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:46:05.332633 30613 raft_consensus.cc:686] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [term 1 LEADER]: Becoming Leader. State: Replica: 5d3f6ec54abf4617b0de37662fc02dac, State: Running, Role: LEADER
I20260619 02:46:05.332717 30613 consensus_queue.cc:237] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } }
I20260619 02:46:05.333549 30131 catalog_manager.cc:5495] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d3f6ec54abf4617b0de37662fc02dac (127.19.113.65). New cstate: current_term: 1 leader_uuid: "5d3f6ec54abf4617b0de37662fc02dac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d3f6ec54abf4617b0de37662fc02dac" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33561 } health_report { overall_health: HEALTHY } } }
W20260619 02:46:05.359457 30324 tablet.cc:2369] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:46:05.370289 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.367930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.19.113.65:33561)
0619 02:46:05.367984 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:05.367987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:05.367996 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:46:05.368036 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:05.368037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:05.368054 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:05.368112 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:46:05.368250 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.368717 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.369506 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:05.369706 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:05.369799 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:05.369960 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:46:05.370062 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:46:05.370132 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:05.370170 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:46:05.370192 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:46:05.371999 30545 raft_consensus.cc:1261] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 1 FOLLOWER]: Refusing update from remote peer 5d3f6ec54abf4617b0de37662fc02dac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:46:05.372004 30411 raft_consensus.cc:1261] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 1 FOLLOWER]: Refusing update from remote peer 5d3f6ec54abf4617b0de37662fc02dac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:46:05.372359 30615 consensus_queue.cc:1035] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [LEADER]: Connected to new peer: Peer: permanent_uuid: "4552174bf13244208b017c94c0e833ed" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:46:05.372480 30613 consensus_queue.cc:1035] T e587cd70b37145cbabcf1151c6423495 P 5d3f6ec54abf4617b0de37662fc02dac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a0381a3314e83a52dfd7e89c2e606" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:46:05.374604 30618 mvcc.cc:205] Tried to move back new op lower bound from 7298405029361180672 to 7298405029203480576. Current Snapshot: MvccSnapshot[applied={T|T < 7298405029361180672}]
I20260619 02:46:05.375010 30619 mvcc.cc:205] Tried to move back new op lower bound from 7298405029361180672 to 7298405029203480576. Current Snapshot: MvccSnapshot[applied={T|T < 7298405029361180672}]
I20260619 02:46:05.375460 30620 mvcc.cc:205] Tried to move back new op lower bound from 7298405029361180672 to 7298405029203480576. Current Snapshot: MvccSnapshot[applied={T|T < 7298405029361180672}]
I20260619 02:46:05.377702 30132 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:46:05.377791 30132 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:46:05.378497 30132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c693ec6fba474a9ca68d72b16f8475ac: Sending DeleteTablet for 3 replicas of tablet e587cd70b37145cbabcf1151c6423495
I20260619 02:46:05.378887 30525 tablet_service.cc:1514] Processing DeleteTablet for tablet e587cd70b37145cbabcf1151c6423495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:46:05 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:49368
I20260619 02:46:05.378950 30391 tablet_service.cc:1514] Processing DeleteTablet for tablet e587cd70b37145cbabcf1151c6423495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:46:05 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:44008
I20260619 02:46:05.379101 30257 tablet_service.cc:1514] Processing DeleteTablet for tablet e587cd70b37145cbabcf1151c6423495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:46:05 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:57962
I20260619 02:46:05.379218 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 30194
I20260619 02:46:05.379912 30627 tablet_replica.cc:330] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: stopping tablet replica
I20260619 02:46:05.380019 30627 raft_consensus.cc:2229] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:46:05.380128 30627 raft_consensus.cc:2258] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:46:05.380277 30628 tablet_replica.cc:330] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: stopping tablet replica
I20260619 02:46:05.380369 30628 raft_consensus.cc:2229] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:46:05.380460 30628 raft_consensus.cc:2258] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:46:05.380591 30627 ts_tablet_manager.cc:1905] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:46:05.380832 30628 ts_tablet_manager.cc:1905] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:46:05.382392 30628 ts_tablet_manager.cc:1918] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:46:05.382428 30627 ts_tablet_manager.cc:1918] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:46:05.382462 30628 log.cc:1196] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/e587cd70b37145cbabcf1151c6423495
I20260619 02:46:05.382489 30627 log.cc:1196] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/e587cd70b37145cbabcf1151c6423495
I20260619 02:46:05.382705 30628 ts_tablet_manager.cc:1939] T e587cd70b37145cbabcf1151c6423495 P 9c2a0381a3314e83a52dfd7e89c2e606: Deleting consensus metadata
I20260619 02:46:05.382740 30627 ts_tablet_manager.cc:1939] T e587cd70b37145cbabcf1151c6423495 P 4552174bf13244208b017c94c0e833ed: Deleting consensus metadata
I20260619 02:46:05.383116 30116 catalog_manager.cc:4841] TS 9c2a0381a3314e83a52dfd7e89c2e606 (127.19.113.66:43389): tablet e587cd70b37145cbabcf1151c6423495 (table test-table [id=f6eb6b2418814fdd8872b7acf896f3ec]) successfully deleted
I20260619 02:46:05.383126 30119 catalog_manager.cc:4841] TS 4552174bf13244208b017c94c0e833ed (127.19.113.67:42767): tablet e587cd70b37145cbabcf1151c6423495 (table test-table [id=f6eb6b2418814fdd8872b7acf896f3ec]) successfully deleted
W20260619 02:46:05.385229 30119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:33561 (error 108)
I20260619 02:46:05.385658 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 30328
I20260619 02:46:05.385696 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:05.385557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:33561 (local address 127.0.0.1:57994)
0619 02:46:05.385620 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:05.385664 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:33561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 02:46:05.385790 30119 catalog_manager.cc:4584] TS 5d3f6ec54abf4617b0de37662fc02dac (127.19.113.65:33561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e587cd70b37145cbabcf1151c6423495: Network error: Client connection negotiation failed: client connection to 127.19.113.65:33561: connect: Connection refused (error 111)
I20260619 02:46:05.390619 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 30462
I20260619 02:46:05.394443 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 30100
2026-06-19T02:46:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1912 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:46:05 dist-test-slave-zqpb krb5kdc[30633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:46:05 dist-test-slave-zqpb krb5kdc[30633](info): set up 2 sockets
Jun 19 02:46:05 dist-test-slave-zqpb krb5kdc[30633](info): commencing operation
krb5kdc: starting...
W20260619 02:46:06.351678 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:46:06 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:46:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:46:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.113.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.113.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/rangeradmin_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/rangeradmin_127.19.113.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.113.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.113.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/rangerlookup_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/rangerlookup_127.19.113.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.113.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.113.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/HTTP_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/HTTP_127.19.113.84@KRBTEST.COM.keytab.
I20260619 02:46:06.511968 19909 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task85DHGI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/postgres -l logfile start
2026-06-19 02:46:08.920 UTC [30663] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:46:08.920 UTC [30663] LOG: listening on IPv4 address "127.19.113.84", port 47457
2026-06-19 02:46:08.924 UTC [30663] LOG: listening on Unix socket "/tmp/.s.PGSQL.47457"
2026-06-19 02:46:08.945 UTC [30666] LOG: database system was shut down at 2026-06-19 02:46:07 UTC
2026-06-19 02:46:08.948 UTC [30663] LOG: database system is ready to accept connections
I20260619 02:46:09.823463 19909 mini_postgres.cc:96] Postgres bound to 47457
2026-06-19 02:46:09.829 UTC [30674] FATAL: database "slave" does not exist
127.19.113.84:47457 - accepting connections
I20260619 02:46:09.829649 19909 mini_ranger.cc:161] Starting Ranger...
I20260619 02:46:09.843516 19909 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:46:10.298620 19909 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:46:10.298707 19909 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-admin
2026-06-19 02:46:10,568 [I] DB FLAVOR :POSTGRES
2026-06-19 02:46:10,569 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:46:10,569 [I] Checking connection..
2026-06-19 02:46:10,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:46:10,816 [I] Checking connection passed.
2026-06-19 02:46:10,816 [I] --------- Verifying version history table ---------
2026-06-19 02:46:10,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:46:11,059 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:46:11,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:46:11,294 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:46:11,295 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:46:11,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:46:11.540 UTC [30738] WARNING: there is no transaction in progress
2026-06-19 02:46:11,546 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:46:11,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:46:11,782 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:46:11,782 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:46:11,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:46:12,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:46:12,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zqpb.c.gcp-upstream.internal','N') ;"
2026-06-19 02:46:12,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:46:12,748 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:46:12,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:46:12,990 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:46:12,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:46:13,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:46:13,503 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:46:13,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:46:14.472 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.489 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.505 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.520 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.536 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.554 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.618 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.625 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.639 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.649 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.659 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.673 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.682 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.689 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.696 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:14.702 UTC [30846] WARNING: there is no transaction in progress
2026-06-19 02:46:15,311 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:46:15,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:46:15,561 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:46:15,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:46:15,817 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:46:15,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:46:16,075 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:46:16,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:46:16,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:46:16,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zqpb.c.gcp-upstream.internal';"
2026-06-19 02:46:16,801 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:46:16,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:46:17,051 [I] DB_PATCHES have already been applied
I20260619 02:46:17.056393 19909 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-admin:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task85DHGI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:46:17.056463 19909 mini_ranger.cc:193] Using host: 127.19.113.84
I20260619 02:46:17.059391 19909 mini_ranger.cc:227] Ranger admin URL: http://127.19.113.84:36527
Jun 19, 2026 2:46:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:46:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:46:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:46:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:46:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:46:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:46:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:46:18.033128 19909 mini_ranger.cc:160] Time spent starting Ranger: real 8.203s user 0.003s sys 0.004s
Jun 19, 2026 2:46:37 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:46:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:46:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:46:37 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:46:38 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:46:39.315286 19909 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.113.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.113.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/rangerkms_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/rangerkms_127.19.113.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.113.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.113.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.113.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/HTTP_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.113.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/HTTP_127.19.113.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:46:39 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837199, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:46:39.391208 19909 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:46:39.399508 19909 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:46:40.241170 19909 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:46:40.241242 19909 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms
I20260619 02:46:40.241261 19909 mini_ranger_kms.cc:227] Using postgres at 127.19.113.84:47457
2026-06-19 02:46:40,401 [I] DB FLAVOR :POSTGRES
2026-06-19 02:46:40,401 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:46:40,402 [I] Checking connection
2026-06-19 02:46:40,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:46:40,641 [I] connection success
2026-06-19 02:46:40,641 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:46:40,641 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:46:40,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:46:40,873 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:46:40,874 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:46:40,874 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:46:40,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:47457/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:46:41,130 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:46:41.573599 19909 mini_ranger_kms.cc:310] Created kms service
I20260619 02:46:41.745965 19909 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:46:41.789228 19909 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:46:41 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837199, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:46:42.177664 19909 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:46:42.177783 19909 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms:/tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:46:42.177804 19909 mini_ranger_kms.cc:240] Using host: 127.19.113.84
I20260619 02:46:42.181335 19909 mini_ranger_kms.cc:276] Ranger KMS PID: 31026
I20260619 02:46:42.181442 19909 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.113.84:43675
Jun 19, 2026 2:46:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:46:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:46:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:46:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:46:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:46:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:46:44.259033 19909 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.868s user 0.000s sys 0.144s
I20260619 02:46:44.259128 19909 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:46:44.259203 19909 mini_ranger_kms.cc:401] 127.19.113.84:43675/kms/v1/keys
Jun 19 02:46:44 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837204, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.113.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:46:44 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837204, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.113.84@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
Jun 19 02:46:45 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837204, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.113.84@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
Jun 19 02:46:47 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:46:47.306041 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36881
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:36881
--ranger_config_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:43675/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:46:47.386307 31077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:47.386524 31077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:47.386544 31077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:47.387799 31077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:46:47.387846 31077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:47.387858 31077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:47.387869 31077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:46:47.387879 31077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:46:47.389523 31077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.113.84:43675/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:36881
--ranger_config_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:47.389746 31077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:47.389986 31077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:47.394755 31083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:47.394929 31085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:47.395020 31082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:47.395151 31077 server_base.cc:1034] running on GCE node
I20260619 02:46:47.395430 31077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:47.395695 31077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:47.396847 31077 hybrid_clock.cc:648] HybridClock initialized: now 1781837207396837 us; error 25 us; skew 500 ppm
Jun 19 02:46:47 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837207, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:46:47.404953 31077 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:46:47.405433 31077 webserver.cc:460] Webserver started at http://127.19.113.126:44673/ using document root <none> and password file <none>
I20260619 02:46:47.405659 31077 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:47.405699 31077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:47.405818 31077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:46:47 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837207, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:46:47.454335 31077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "cf1ae70f81644576988379804c8b1619"
format_stamp: "Formatted at 2026-06-19 02:46:47 on dist-test-slave-zqpb"
server_key: "71be1415419638bdde9981288690d72f"
server_key_iv: "fe03e9a5909972eae2d2839a4757fd30"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:47.454784 31077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "cf1ae70f81644576988379804c8b1619"
format_stamp: "Formatted at 2026-06-19 02:46:47 on dist-test-slave-zqpb"
server_key: "71be1415419638bdde9981288690d72f"
server_key_iv: "fe03e9a5909972eae2d2839a4757fd30"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:47.456578 31077 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 02:46:47.468906 31092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:47.469103 31077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:46:47.469225 31077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "cf1ae70f81644576988379804c8b1619"
format_stamp: "Formatted at 2026-06-19 02:46:47 on dist-test-slave-zqpb"
server_key: "71be1415419638bdde9981288690d72f"
server_key_iv: "fe03e9a5909972eae2d2839a4757fd30"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:47.469327 31077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:47.496054 31077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:47.497931 31077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:47.498071 31077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:47.502790 31077 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:36881
I20260619 02:46:47.502811 31144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:36881 every 8 connection(s)
I20260619 02:46:47.503218 31077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:46:47.503872 31145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:47.506117 31145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619: Bootstrap starting.
I20260619 02:46:47.507113 31145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:47.507375 31145 log.cc:824] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:47.508095 31145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619: No bootstrap required, opened a new log
I20260619 02:46:47.510144 31145 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1ae70f81644576988379804c8b1619" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36881 } }
I20260619 02:46:47.510294 31145 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:47.510317 31145 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1ae70f81644576988379804c8b1619, State: Initialized, Role: FOLLOWER
I20260619 02:46:47.510444 31145 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1ae70f81644576988379804c8b1619" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36881 } }
I20260619 02:46:47.510512 31145 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:46:47.510537 31145 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:46:47.510581 31145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:47.511183 31145 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1ae70f81644576988379804c8b1619" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36881 } }
I20260619 02:46:47.511271 31145 leader_election.cc:302] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf1ae70f81644576988379804c8b1619; no voters:
I20260619 02:46:47.511427 31145 leader_election.cc:288] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:46:47.511466 31148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:46:47.511631 31148 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [term 1 LEADER]: Becoming Leader. State: Replica: cf1ae70f81644576988379804c8b1619, State: Running, Role: LEADER
I20260619 02:46:47.511693 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 31077
I20260619 02:46:47.511801 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:46:47.511773 31148 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1ae70f81644576988379804c8b1619" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36881 } }
I20260619 02:46:47.511996 19909 external_mini_cluster.cc:1428] Setting key 5b943e3f6bbc1297f4b3ab02acbafd05
I20260619 02:46:47.512163 31148 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf1ae70f81644576988379804c8b1619. Latest consensus state: current_term: 1 leader_uuid: "cf1ae70f81644576988379804c8b1619" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1ae70f81644576988379804c8b1619" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36881 } } }
I20260619 02:46:47.512552 31149 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf1ae70f81644576988379804c8b1619" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1ae70f81644576988379804c8b1619" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36881 } } }
I20260619 02:46:47.512652 31145 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:46:47.512745 31148 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [sys.catalog]: This master's current role is: LEADER
I20260619 02:46:47.513518 31154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:46:47.514025 31154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:46:47.514317 31145 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:46:47.514637 31149 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619 [sys.catalog]: This master's current role is: LEADER
Jun 19 02:46:47 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:46:47.526024 31153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:47.512451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.19.113.126:36881)
0619 02:46:47.512741 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:47.512751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:47.513072 (+ 321us) server_negotiation.cc:407] Connection header received
0619 02:46:47.513213 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:47.513219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:47.513280 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:47.513413 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:47.514135 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:47.517384 (+ 3249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:47.518418 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:47.519297 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:47.519418 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:47.522293 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:47.522298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:47.522298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:47.522311 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:47.524327 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:47.524728 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:47.524730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:47.524731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:47.524878 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:47.525074 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:47.525075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:47.525075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:47.525392 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:47.525535 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:47.525647 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:46:47.525775 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":60,"threads_started":1}
I20260619 02:46:47.527659 31154 catalog_manager.cc:1347] Generated new cluster ID: 86ab007022c345998fdeaebeb725eebe
I20260619 02:46:47.527726 31154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:46:47.552007 31154 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:46:47.552822 31154 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:46:47.562911 31154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619: Generated new TSK 0
I20260619 02:46:47.563199 31154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:46:48.285384 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:43675/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36881
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:46:48.369143 31190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:48.369423 31190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:48.369458 31190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:48.370716 31190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:48.370759 31190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:48.370793 31190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:46:48.372430 31190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.113.84:43675/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:48.372653 31190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:48.372864 31190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:48.377666 31198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:48.377656 31195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:48.377676 31196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:48.378023 31190 server_base.cc:1034] running on GCE node
I20260619 02:46:48.378209 31190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:48.378435 31190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:48.379573 31190 hybrid_clock.cc:648] HybridClock initialized: now 1781837208379555 us; error 34 us; skew 500 ppm
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:46:48.381825 31190 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:46:48.382251 31190 webserver.cc:460] Webserver started at http://127.19.113.65:42691/ using document root <none> and password file <none>
I20260619 02:46:48.382462 31190 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:48.382503 31190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:48.382619 31190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:46:48.401319 31190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "4eaf285eb32f49d1a4a3354e43da7b16"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "23503147d562dbe2507bdffce4f8ed1f"
server_key_iv: "2062cf9c2f6086cdb05802fcc69439ec"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.401695 31190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "4eaf285eb32f49d1a4a3354e43da7b16"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "23503147d562dbe2507bdffce4f8ed1f"
server_key_iv: "2062cf9c2f6086cdb05802fcc69439ec"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.403153 31190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:46:48.417515 31205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:48.417719 31190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260619 02:46:48.417802 31190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "4eaf285eb32f49d1a4a3354e43da7b16"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "23503147d562dbe2507bdffce4f8ed1f"
server_key_iv: "2062cf9c2f6086cdb05802fcc69439ec"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.417892 31190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:48.442222 31190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:48.444152 31190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:48.444291 31190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:48.444506 31190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:48.444844 31190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:48.444880 31190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:48.444922 31190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:48.444944 31190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:48.452270 31190 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:45231
I20260619 02:46:48.452376 31320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:45231 every 8 connection(s)
I20260619 02:46:48.452646 31190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:46:48.460599 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 31190
I20260619 02:46:48.460712 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:46:48.460909 19909 external_mini_cluster.cc:1428] Setting key 097a1b6dff48f1c87a51f5d6ced2c735
I20260619 02:46:48.461494 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:48.453349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:54695 (local address 127.19.113.126:36881)
0619 02:46:48.453502 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:48.453504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:48.453565 (+ 61us) server_negotiation.cc:407] Connection header received
0619 02:46:48.454097 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:48.454098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:48.454112 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:48.454173 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:48.454701 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:48.455271 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:48.456214 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:48.456425 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:48.456477 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:48.458349 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:48.458350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:48.458351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:48.458358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:48.459680 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:48.460033 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:48.460034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:48.460034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:48.460089 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:48.460299 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:48.460300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:48.460300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:48.460441 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:48.460499 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:48.461255 (+ 756us) server_negotiation.cc:299] Negotiation successful
0619 02:46:48.461336 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260619 02:46:48.461731 31323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:48.453288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36881 (local address 127.19.113.65:54695)
0619 02:46:48.453440 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:48.453484 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:48.453999 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:48.454240 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:48.454244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:48.454430 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:48.454614 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:48.454618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:48.455350 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:48.455351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:48.456134 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:48.456136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:48.456490 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:48.456491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:48.456605 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:48.457079 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:48.457087 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:48.458263 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:48.459753 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:48.459759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:48.459762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:48.459971 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:48.460141 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:48.460141 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:48.460142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:48.460247 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:48.460526 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:48.460533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:48.460697 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 02:46:48.460800 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 02:46:48.460946 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260619 02:46:48.462136 31321 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36881
I20260619 02:46:48.462222 31321 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:48.462395 31321 heartbeater.cc:511] Master 127.19.113.126:36881 requested a full tablet report, sending...
I20260619 02:46:48.462919 31109 ts_manager.cc:194] Registered new tserver with Master: 4eaf285eb32f49d1a4a3354e43da7b16 (127.19.113.65:45231)
I20260619 02:46:48.463734 31109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:54695
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:46:48.489253 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:43675/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36881
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:46:48.606374 31327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:48.606709 31327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:48.606761 31327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:48.608451 31327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:48.608506 31327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:48.608549 31327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:46:48.610695 31327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.113.84:43675/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:48.610967 31327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:48.611267 31327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 02:46:48.617409 31337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:48.617412 31334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:48.617766 31335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:48.619006 31327 server_base.cc:1034] running on GCE node
I20260619 02:46:48.619248 31327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:48.619534 31327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:48.620740 31327 hybrid_clock.cc:648] HybridClock initialized: now 1781837208620712 us; error 32 us; skew 500 ppm
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:46:48.623914 31327 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:46:48.624524 31327 webserver.cc:460] Webserver started at http://127.19.113.66:39965/ using document root <none> and password file <none>
I20260619 02:46:48.624836 31327 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:48.624944 31327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:48.625104 31327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:46:48.641831 31327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "24b11e0364cb47c8bc9c8929ef55ec0c"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "87ed117c0adf0c45f5f40ddc3014c80c"
server_key_iv: "e9992087d2196424b54adfd5907470b0"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.642271 31327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "24b11e0364cb47c8bc9c8929ef55ec0c"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "87ed117c0adf0c45f5f40ddc3014c80c"
server_key_iv: "e9992087d2196424b54adfd5907470b0"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.645215 31327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260619 02:46:48.666096 31344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:48.666287 31327 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260619 02:46:48.666373 31327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "24b11e0364cb47c8bc9c8929ef55ec0c"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "87ed117c0adf0c45f5f40ddc3014c80c"
server_key_iv: "e9992087d2196424b54adfd5907470b0"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.666491 31327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:48.713972 31327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:48.716214 31327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:48.716357 31327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:48.716614 31327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:48.716982 31327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:48.717034 31327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:48.717070 31327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:48.717092 31327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:48.723802 31327 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:44827
I20260619 02:46:48.724224 31327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:46:48.726917 31458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:44827 every 8 connection(s)
I20260619 02:46:48.733459 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 31327
I20260619 02:46:48.733579 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:46:48.733776 19909 external_mini_cluster.cc:1428] Setting key adc73b5620f5266fdfde27f61a3ee226
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:46:48.738131 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:48.727901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:60729 (local address 127.19.113.126:36881)
0619 02:46:48.727951 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:48.727953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:48.727964 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:46:48.728587 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:48.728589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:48.728605 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:48.728636 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:48.729498 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:48.730236 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:48.731744 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:48.732045 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:48.732090 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:48.734820 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:48.734821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:48.734822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:48.734829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:48.736488 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:48.736991 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:48.736992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:48.736992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:48.737053 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:48.737369 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:48.737370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:48.737370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:48.737516 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:48.737559 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:48.737946 (+ 387us) server_negotiation.cc:299] Negotiation successful
0619 02:46:48.738037 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:46:48.738916 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:48.727592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36881 (local address 127.19.113.66:60729)
0619 02:46:48.727784 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:48.727829 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:48.728455 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:48.728824 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:48.728829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:48.729070 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:48.729374 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:48.729390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:48.730349 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:48.730350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:48.731638 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:48.731642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:48.732185 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:48.732186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:48.732334 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:48.732938 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:48.732948 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:48.734709 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:48.736586 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:48.736594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:48.736598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:48.736897 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:48.737126 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:48.737126 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:48.737127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:48.737275 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:48.737609 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:48.737616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:48.737782 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 02:46:48.738490 (+ 708us) client_negotiation.cc:240] Negotiation successful
0619 02:46:48.738656 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260619 02:46:48.739670 31459 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36881
I20260619 02:46:48.739763 31459 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:48.739944 31459 heartbeater.cc:511] Master 127.19.113.126:36881 requested a full tablet report, sending...
I20260619 02:46:48.740360 31109 ts_manager.cc:194] Registered new tserver with Master: 24b11e0364cb47c8bc9c8929ef55ec0c (127.19.113.66:44827)
I20260619 02:46:48.740840 31109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:60729
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:46:48.775089 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:43675/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36881
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:46:48.918911 31464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:46:48.919198 31464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:46:48.919230 31464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:46:48.920923 31464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:46:48.920984 31464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:46:48.921020 31464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:46:48.923190 31464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.113.84:43675/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:46:48.923451 31464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:46:48.923712 31464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:46:48.930212 31470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:48.930250 31472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:46:48.930444 31469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:46:48.931010 31464 server_base.cc:1034] running on GCE node
I20260619 02:46:48.931224 31464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:46:48.931507 31464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:46:48.932674 31464 hybrid_clock.cc:648] HybridClock initialized: now 1781837208932658 us; error 37 us; skew 500 ppm
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:46:48.936070 31464 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:46:48.936695 31464 webserver.cc:460] Webserver started at http://127.19.113.67:38713/ using document root <none> and password file <none>
I20260619 02:46:48.936990 31464 fs_manager.cc:359] Metadata directory not provided
I20260619 02:46:48.937055 31464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:46:48.937199 31464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:46:48 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:46:48.959254 31464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "af817aae7febfa497a76dd02d46a3ead"
server_key_iv: "f8b80dd42d6c1ba83e4019cd53a05fd7"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.959774 31464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "af817aae7febfa497a76dd02d46a3ead"
server_key_iv: "f8b80dd42d6c1ba83e4019cd53a05fd7"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.962911 31464 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:46:48.986995 31479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:48.987190 31464 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260619 02:46:48.987287 31464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b"
format_stamp: "Formatted at 2026-06-19 02:46:48 on dist-test-slave-zqpb"
server_key: "af817aae7febfa497a76dd02d46a3ead"
server_key_iv: "f8b80dd42d6c1ba83e4019cd53a05fd7"
server_key_version: "kuduclusterkey@0"
I20260619 02:46:48.987386 31464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:46:48.998929 31464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:46:49.001226 31464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:46:49.001384 31464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:46:49.002138 31464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:46:49.002549 31464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:46:49.002604 31464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:49.002638 31464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:46:49.002655 31464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:46:49.008582 31464 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:42873
I20260619 02:46:49.009017 31464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:46:49.012993 31592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:42873 every 8 connection(s)
I20260619 02:46:49.017958 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 31464
I20260619 02:46:49.018059 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:46:49.018245 19909 external_mini_cluster.cc:1428] Setting key 85ab508455c1d063505cf728fe401487
Jun 19 02:46:49 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:46:49.023911 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.012695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:36339 (local address 127.19.113.126:36881)
0619 02:46:49.012739 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:49.012740 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:49.013308 (+ 568us) server_negotiation.cc:407] Connection header received
0619 02:46:49.014030 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:49.014032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:49.014046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:49.014075 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:49.014913 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.015683 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.017159 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.017462 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.017500 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.020686 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:49.020687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:49.020688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:49.020696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:49.022371 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:49.022853 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:49.022854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:49.022854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:49.022931 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:49.023189 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:49.023190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:49.023191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:49.023315 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:49.023354 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:49.023707 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 02:46:49.023794 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:46:49.024641 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.012323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36881 (local address 127.19.113.67:36339)
0619 02:46:49.013185 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:49.013227 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:49.013910 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:49.014254 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:49.014258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:49.014492 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:46:49.014768 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.014774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.015792 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.015793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.017029 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.017032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.017584 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.017585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.017723 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.018372 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:46:49.018381 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:46:49.020576 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:46:49.022463 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:49.022471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:49.022474 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:49.022759 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:49.022998 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:46:49.022999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:46:49.022999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:46:49.023126 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:46:49.023396 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:46:49.023402 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:46:49.023560 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 02:46:49.024240 (+ 680us) client_negotiation.cc:240] Negotiation successful
0619 02:46:49.024405 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":84,"threads_started":1}
I20260619 02:46:49.025425 31593 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36881
I20260619 02:46:49.025520 31593 heartbeater.cc:464] Registering TS with master...
I20260619 02:46:49.025697 31593 heartbeater.cc:511] Master 127.19.113.126:36881 requested a full tablet report, sending...
I20260619 02:46:49.026103 31109 ts_manager.cc:194] Registered new tserver with Master: d0a677a2a83a41e0b9a963bfd8e31f3b (127.19.113.67:42873)
I20260619 02:46:49.026588 31109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:36339
I20260619 02:46:49.029681 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:46:49.039229 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.031684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.19.113.126:36881)
0619 02:46:49.031721 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:49.031723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:49.032035 (+ 312us) server_negotiation.cc:407] Connection header received
0619 02:46:49.032121 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:49.032122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:49.032131 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:49.032157 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:46:49.032857 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.033609 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.034695 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.035013 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.035052 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.036455 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:46:49.036457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:46:49.036457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:46:49.036464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:46:49.038038 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:49.038420 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:49.038421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:49.038422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:49.038474 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:46:49.038647 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:46:49.038648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:46:49.038648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:46:49.038766 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:46:49.038834 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:49.039065 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 02:46:49.039136 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:46:49.041226 31109 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:46:49.043454 31109 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:46:49.057796 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.049864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:45231 (local address 127.0.0.1:33160)
0619 02:46:49.050864 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:49.050888 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:49.050930 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:49.051201 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:49.051203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:49.051206 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:49.051280 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.051282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.052853 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.052854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.054233 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.054236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.057440 (+ 3204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.057441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.057552 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.057569 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 02:46:49.057632 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 02:46:49.057695 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"thread_start_us":60,"threads_started":1}
I20260619 02:46:49.058269 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.050143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.19.113.65:45231)
0619 02:46:49.050487 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:49.050492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:49.050988 (+ 496us) server_negotiation.cc:407] Connection header received
0619 02:46:49.051028 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:49.051030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:49.051047 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:49.051116 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:49.051374 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.052756 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.054597 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.055467 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.055518 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.055582 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:49.058135 (+ 2553us) server_negotiation.cc:299] Negotiation successful
0619 02:46:49.058188 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":66,"threads_started":1}
I20260619 02:46:49.059325 31255 tablet_service.cc:1467] Processing CreateTablet for tablet f567d567052c44b98d80180be53c6a5a (DEFAULT_TABLE table=test-table [id=6efb35310ab340dcafcf5c7b3ae8fd59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:46:49.059708 31255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f567d567052c44b98d80180be53c6a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:49.066685 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.049475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:42873 (local address 127.0.0.1:34406)
0619 02:46:49.050553 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:49.050596 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:49.050658 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:49.055988 (+ 5330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:49.055989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:49.055992 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:49.056075 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.056078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.064030 (+ 7952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.064031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.065143 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.065147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.066446 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.066447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.066521 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.066526 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:46:49.066554 (+ 28us) client_negotiation.cc:240] Negotiation successful
0619 02:46:49.066600 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"thread_start_us":79,"threads_started":1}
I20260619 02:46:49.067898 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.049558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.19.113.67:42873)
0619 02:46:49.055688 (+ 6130us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:49.055693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:49.055703 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:46:49.055796 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:49.055799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:49.055817 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:49.056157 (+ 340us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:49.056241 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.063928 (+ 7687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.065276 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.066091 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.067631 (+ 1540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.067731 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:49.067771 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 02:46:49.067816 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6084,"thread_start_us":85,"threads_started":1}
I20260619 02:46:49.070168 31526 tablet_service.cc:1467] Processing CreateTablet for tablet f567d567052c44b98d80180be53c6a5a (DEFAULT_TABLE table=test-table [id=6efb35310ab340dcafcf5c7b3ae8fd59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:46:49.070525 31526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f567d567052c44b98d80180be53c6a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:49.074553 31611 tablet_bootstrap.cc:492] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16: Bootstrap starting.
I20260619 02:46:49.075376 31611 tablet_bootstrap.cc:654] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:49.078034 31611 log.cc:824] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:49.078636 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.049703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:44827 (local address 127.0.0.1:42710)
0619 02:46:49.050759 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:49.050767 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:49.050800 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:49.068131 (+ 17331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:49.068132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:49.068135 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:49.068178 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.068180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.075529 (+ 7349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.075530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.076718 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.076721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.078401 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.078402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.078476 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.078482 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:46:49.078511 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 02:46:49.078556 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"thread_start_us":60,"threads_started":1}
I20260619 02:46:49.079247 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.050401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.19.113.66:44827)
0619 02:46:49.066245 (+ 15844us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:49.066251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:49.066260 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:46:49.066312 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:49.066314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:49.066331 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:49.068258 (+ 1927us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:49.068354 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.069468 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.076846 (+ 7378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.077664 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.078993 (+ 1329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.079095 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:49.079124 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 02:46:49.079164 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15800,"thread_start_us":88,"threads_started":1}
I20260619 02:46:49.079550 31612 tablet_bootstrap.cc:492] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: Bootstrap starting.
I20260619 02:46:49.080245 31393 tablet_service.cc:1467] Processing CreateTablet for tablet f567d567052c44b98d80180be53c6a5a (DEFAULT_TABLE table=test-table [id=6efb35310ab340dcafcf5c7b3ae8fd59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:46:49.080585 31393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f567d567052c44b98d80180be53c6a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:46:49.081189 31612 tablet_bootstrap.cc:654] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:49.081547 31612 log.cc:824] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:49.082386 31611 tablet_bootstrap.cc:492] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16: No bootstrap required, opened a new log
I20260619 02:46:49.082500 31611 ts_tablet_manager.cc:1397] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260619 02:46:49.085772 31611 raft_consensus.cc:348] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.086092 31611 raft_consensus.cc:374] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:49.086182 31611 raft_consensus.cc:729] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaf285eb32f49d1a4a3354e43da7b16, State: Initialized, Role: FOLLOWER
I20260619 02:46:49.086405 31611 consensus_queue.cc:260] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.086799 31611 ts_tablet_manager.cc:1428] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 02:46:49.086978 31321 heartbeater.cc:503] Master 127.19.113.126:36881 was elected leader, sending a full tablet report...
I20260619 02:46:49.088572 31612 tablet_bootstrap.cc:492] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: No bootstrap required, opened a new log
I20260619 02:46:49.089038 31612 ts_tablet_manager.cc:1397] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260619 02:46:49.089879 31615 tablet_bootstrap.cc:492] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: Bootstrap starting.
I20260619 02:46:49.090916 31615 tablet_bootstrap.cc:654] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: Neither blocks nor log segments found. Creating new log.
I20260619 02:46:49.091233 31615 log.cc:824] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:46:49.091394 31612 raft_consensus.cc:348] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.091609 31612 raft_consensus.cc:374] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:49.091687 31612 raft_consensus.cc:729] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a677a2a83a41e0b9a963bfd8e31f3b, State: Initialized, Role: FOLLOWER
I20260619 02:46:49.091883 31612 consensus_queue.cc:260] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.092252 31612 ts_tablet_manager.cc:1428] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:46:49.092418 31593 heartbeater.cc:503] Master 127.19.113.126:36881 was elected leader, sending a full tablet report...
I20260619 02:46:49.094659 31615 tablet_bootstrap.cc:492] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: No bootstrap required, opened a new log
I20260619 02:46:49.094779 31615 ts_tablet_manager.cc:1397] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260619 02:46:49.096840 31615 raft_consensus.cc:348] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.097090 31615 raft_consensus.cc:374] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:46:49.097178 31615 raft_consensus.cc:729] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24b11e0364cb47c8bc9c8929ef55ec0c, State: Initialized, Role: FOLLOWER
I20260619 02:46:49.097380 31615 consensus_queue.cc:260] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.097709 31615 ts_tablet_manager.cc:1428] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:46:49.097936 31459 heartbeater.cc:503] Master 127.19.113.126:36881 was elected leader, sending a full tablet report...
Jun 19 02:46:49 dist-test-slave-zqpb krb5kdc[30633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:46:49.176319 31616 raft_consensus.cc:482] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:46:49.176467 31616 raft_consensus.cc:504] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.176834 31616 leader_election.cc:288] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0a677a2a83a41e0b9a963bfd8e31f3b (127.19.113.67:42873), 24b11e0364cb47c8bc9c8929ef55ec0c (127.19.113.66:44827)
I20260619 02:46:49.180805 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.177363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:37241 (local address 127.19.113.67:42873)
0619 02:46:49.177405 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:49.177407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:49.177414 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:46:49.177439 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:49.177439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:49.177450 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:49.177491 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:49.177744 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.178469 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.179941 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.180458 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.180500 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.180509 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:49.180696 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 02:46:49.180734 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:46:49.181231 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.176987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:42873 (local address 127.19.113.65:37241)
0619 02:46:49.177153 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:49.177166 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:49.177223 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:49.177579 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:49.177580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:49.177597 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:49.177673 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.177675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.178584 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.178585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.179843 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.179846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.180560 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.180561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.180627 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.180631 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:46:49.181110 (+ 479us) client_negotiation.cc:240] Negotiation successful
0619 02:46:49.181155 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260619 02:46:49.181663 31547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f567d567052c44b98d80180be53c6a5a" candidate_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" is_pre_election: true
I20260619 02:46:49.181814 31547 raft_consensus.cc:2454] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eaf285eb32f49d1a4a3354e43da7b16 in term 0.
I20260619 02:46:49.182112 31209 leader_election.cc:302] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaf285eb32f49d1a4a3354e43da7b16, d0a677a2a83a41e0b9a963bfd8e31f3b; no voters:
I20260619 02:46:49.182233 31616 raft_consensus.cc:2781] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:46:49.182266 31616 raft_consensus.cc:482] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:46:49.182283 31616 raft_consensus.cc:3037] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:49.183139 31616 raft_consensus.cc:504] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.183257 31616 leader_election.cc:288] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [CANDIDATE]: Term 1 election: Requested vote from peers d0a677a2a83a41e0b9a963bfd8e31f3b (127.19.113.67:42873), 24b11e0364cb47c8bc9c8929ef55ec0c (127.19.113.66:44827)
I20260619 02:46:49.183760 31547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f567d567052c44b98d80180be53c6a5a" candidate_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b"
I20260619 02:46:49.183836 31547 raft_consensus.cc:3037] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:49.184641 31547 raft_consensus.cc:2454] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eaf285eb32f49d1a4a3354e43da7b16 in term 1.
I20260619 02:46:49.184863 31209 leader_election.cc:302] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaf285eb32f49d1a4a3354e43da7b16, d0a677a2a83a41e0b9a963bfd8e31f3b; no voters:
I20260619 02:46:49.184978 31616 raft_consensus.cc:2781] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:46:49.185113 31616 raft_consensus.cc:686] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [term 1 LEADER]: Becoming Leader. State: Replica: 4eaf285eb32f49d1a4a3354e43da7b16, State: Running, Role: LEADER
I20260619 02:46:49.185200 31616 consensus_queue.cc:237] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } }
I20260619 02:46:49.186033 31109 catalog_manager.cc:5495] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4eaf285eb32f49d1a4a3354e43da7b16 (127.19.113.65). New cstate: current_term: 1 leader_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 } health_report { overall_health: UNKNOWN } } }
I20260619 02:46:49.191089 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.177404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:47653 (local address 127.19.113.66:44827)
0619 02:46:49.177549 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:49.177551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:49.178702 (+ 1151us) server_negotiation.cc:407] Connection header received
0619 02:46:49.182936 (+ 4234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:49.182938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:49.182954 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:49.182987 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:46:49.183186 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.183838 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.190060 (+ 6222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.190656 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.190730 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.190739 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:49.190980 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 02:46:49.191021 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260619 02:46:49.191116 31621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.177197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:44827 (local address 127.19.113.65:47653)
0619 02:46:49.178631 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:49.178642 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:46:49.179640 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:46:49.183056 (+ 3416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:46:49.183057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:46:49.183059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:46:49.183116 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.183118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.185705 (+ 2587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.185706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.186758 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:46:49.186761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.190774 (+ 4013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.190775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:46:49.190849 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.190908 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 02:46:49.190962 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:46:49.191010 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1415,"thread_start_us":59,"threads_started":1}
I20260619 02:46:49.191423 31410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f567d567052c44b98d80180be53c6a5a" candidate_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" is_pre_election: true
I20260619 02:46:49.191620 31410 raft_consensus.cc:2454] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eaf285eb32f49d1a4a3354e43da7b16 in term 0.
I20260619 02:46:49.191751 31409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f567d567052c44b98d80180be53c6a5a" candidate_uuid: "4eaf285eb32f49d1a4a3354e43da7b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c"
I20260619 02:46:49.191818 31409 raft_consensus.cc:3037] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:46:49.192648 31409 raft_consensus.cc:2454] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eaf285eb32f49d1a4a3354e43da7b16 in term 1.
W20260619 02:46:49.203485 31322 tablet.cc:2369] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:46:49.206506 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.203263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33166 (local address 127.19.113.65:45231)
0619 02:46:49.203315 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:46:49.203317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:46:49.203357 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:46:49.203399 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:46:49.203400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:46:49.203420 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:46:49.203482 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:46:49.203662 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.204387 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.205412 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:46:49.205708 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:46:49.205759 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:46:49.205973 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:46:49.206100 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:46:49.206167 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:46:49.206394 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 02:46:49.206417 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:46:49.208531 31547 raft_consensus.cc:1261] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 1 FOLLOWER]: Refusing update from remote peer 4eaf285eb32f49d1a4a3354e43da7b16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:46:49.208578 31409 raft_consensus.cc:1261] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 1 FOLLOWER]: Refusing update from remote peer 4eaf285eb32f49d1a4a3354e43da7b16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:46:49.208875 31616 consensus_queue.cc:1035] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24b11e0364cb47c8bc9c8929ef55ec0c" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:46:49.208982 31622 consensus_queue.cc:1035] T f567d567052c44b98d80180be53c6a5a P 4eaf285eb32f49d1a4a3354e43da7b16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0a677a2a83a41e0b9a963bfd8e31f3b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 42873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:46:49.211632 31625 mvcc.cc:205] Tried to move back new op lower bound from 7298405208915021824 to 7298405208823128064. Current Snapshot: MvccSnapshot[applied={T|T < 7298405208915021824}]
I20260619 02:46:49.214197 31626 mvcc.cc:205] Tried to move back new op lower bound from 7298405208915021824 to 7298405208823128064. Current Snapshot: MvccSnapshot[applied={T|T < 7298405208915021824}]
I20260619 02:46:49.214810 31627 mvcc.cc:205] Tried to move back new op lower bound from 7298405208915021824 to 7298405208823128064. Current Snapshot: MvccSnapshot[applied={T|T < 7298405208915021824}]
I20260619 02:46:49.217217 31108 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:46:49.217325 31108 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:46:49.218127 31108 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf1ae70f81644576988379804c8b1619: Sending DeleteTablet for 3 replicas of tablet f567d567052c44b98d80180be53c6a5a
I20260619 02:46:49.218565 31526 tablet_service.cc:1514] Processing DeleteTablet for tablet f567d567052c44b98d80180be53c6a5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:46:49 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:34406
I20260619 02:46:49.218783 31393 tablet_service.cc:1514] Processing DeleteTablet for tablet f567d567052c44b98d80180be53c6a5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:46:49 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:42710
I20260619 02:46:49.219187 31255 tablet_service.cc:1514] Processing DeleteTablet for tablet f567d567052c44b98d80180be53c6a5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:46:49 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:33160
I20260619 02:46:49.219787 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 31190
I20260619 02:46:49.220065 31635 tablet_replica.cc:330] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: stopping tablet replica
I20260619 02:46:49.220163 31635 raft_consensus.cc:2229] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:46:49.220259 31635 raft_consensus.cc:2258] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:46:49.220407 31634 tablet_replica.cc:330] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: stopping tablet replica
I20260619 02:46:49.220505 31634 raft_consensus.cc:2229] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:46:49.220569 31634 raft_consensus.cc:2258] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:46:49.224541 31635 ts_tablet_manager.cc:1905] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:46:49.225018 31634 ts_tablet_manager.cc:1905] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:46:49.226742 31634 ts_tablet_manager.cc:1918] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:46:49.228492 31635 ts_tablet_manager.cc:1918] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:46:49.232239 31635 log.cc:1196] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/f567d567052c44b98d80180be53c6a5a
I20260619 02:46:49.232424 31634 log.cc:1196] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/f567d567052c44b98d80180be53c6a5a
I20260619 02:46:49.232800 31635 ts_tablet_manager.cc:1939] T f567d567052c44b98d80180be53c6a5a P 24b11e0364cb47c8bc9c8929ef55ec0c: Deleting consensus metadata
I20260619 02:46:49.233445 31634 ts_tablet_manager.cc:1939] T f567d567052c44b98d80180be53c6a5a P d0a677a2a83a41e0b9a963bfd8e31f3b: Deleting consensus metadata
I20260619 02:46:49.233995 31095 catalog_manager.cc:4841] TS 24b11e0364cb47c8bc9c8929ef55ec0c (127.19.113.66:44827): tablet f567d567052c44b98d80180be53c6a5a (table test-table [id=6efb35310ab340dcafcf5c7b3ae8fd59]) successfully deleted
I20260619 02:46:49.234578 31095 catalog_manager.cc:4841] TS d0a677a2a83a41e0b9a963bfd8e31f3b (127.19.113.67:42873): tablet f567d567052c44b98d80180be53c6a5a (table test-table [id=6efb35310ab340dcafcf5c7b3ae8fd59]) successfully deleted
W20260619 02:46:49.235040 31094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:45231 (error 108)
I20260619 02:46:49.235740 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 31327
I20260619 02:46:49.244026 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 31464
I20260619 02:46:49.250196 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:46:49.250064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:45231 (local address 127.0.0.1:33168)
0619 02:46:49.250108 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:46:49.250153 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 02:46:49.250568 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 31077
I20260619 02:46:49.255048 19909 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:46:49.589401 19909 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:46:49.589514 19909 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:46:49.923915 19909 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:46:49.924 UTC [30663] LOG: received smart shutdown request
2026-06-19 02:46:49.927 UTC [30663] LOG: background worker "logical replication launcher" (PID 30672) exited with exit code 1
2026-06-19 02:46:49.927 UTC [30667] LOG: shutting down
2026-06-19 02:46:50.080 UTC [30663] LOG: database system is shut down
2026-06-19T02:46:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44773 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:46:50 dist-test-slave-zqpb krb5kdc[31646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:46:50 dist-test-slave-zqpb krb5kdc[31646](info): set up 2 sockets
Jun 19 02:46:50 dist-test-slave-zqpb krb5kdc[31646](info): commencing operation
krb5kdc: starting...
W20260619 02:46:51.134646 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:46:51 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:46:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.113.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.113.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/rangeradmin_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/rangeradmin_127.19.113.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.113.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.113.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/rangerlookup_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/rangerlookup_127.19.113.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.113.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.113.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/HTTP_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/HTTP_127.19.113.84@KRBTEST.COM.keytab.
I20260619 02:46:51.287490 19909 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task85DHGI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/postgres -l logfile start
2026-06-19 02:46:53.586 UTC [31676] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:46:53.586 UTC [31676] LOG: listening on IPv4 address "127.19.113.84", port 53323
2026-06-19 02:46:53.590 UTC [31676] LOG: listening on Unix socket "/tmp/.s.PGSQL.53323"
2026-06-19 02:46:53.609 UTC [31679] LOG: database system was shut down at 2026-06-19 02:46:52 UTC
2026-06-19 02:46:53.613 UTC [31676] LOG: database system is ready to accept connections
I20260619 02:46:54.486531 19909 mini_postgres.cc:96] Postgres bound to 53323
2026-06-19 02:46:54.492 UTC [31687] FATAL: database "slave" does not exist
127.19.113.84:53323 - accepting connections
I20260619 02:46:54.492445 19909 mini_ranger.cc:161] Starting Ranger...
I20260619 02:46:54.506206 19909 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:46:54.909593 19909 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:46:54.909684 19909 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-admin
2026-06-19 02:46:55,168 [I] DB FLAVOR :POSTGRES
2026-06-19 02:46:55,168 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:46:55,168 [I] Checking connection..
2026-06-19 02:46:55,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:46:55,403 [I] Checking connection passed.
2026-06-19 02:46:55,403 [I] --------- Verifying version history table ---------
2026-06-19 02:46:55,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:46:55,640 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:46:55,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:46:55,866 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:46:55,866 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:46:55,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:46:56.113 UTC [31751] WARNING: there is no transaction in progress
2026-06-19 02:46:56,119 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:46:56,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:46:56,377 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:46:56,377 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:46:56,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:46:56,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:46:56,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zqpb.c.gcp-upstream.internal','N') ;"
2026-06-19 02:46:57,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:46:57,324 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:46:57,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:46:57,567 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:46:57,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:46:57,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:46:58,045 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:46:58,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:46:58.959 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:58.974 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:58.990 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.005 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.023 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.040 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.094 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.101 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.115 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.124 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.134 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.148 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.157 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.163 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.169 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59.175 UTC [31859] WARNING: there is no transaction in progress
2026-06-19 02:46:59,776 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:46:59,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:46:59,997 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:46:59,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:47:00,257 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:47:00,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:47:00,514 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:47:00,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:47:00,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:47:00,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zqpb.c.gcp-upstream.internal';"
2026-06-19 02:47:01,233 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:47:01,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:47:01,479 [I] DB_PATCHES have already been applied
I20260619 02:47:01.485332 19909 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-admin:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task85DHGI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:47:01.485409 19909 mini_ranger.cc:193] Using host: 127.19.113.84
I20260619 02:47:01.488358 19909 mini_ranger.cc:227] Ranger admin URL: http://127.19.113.84:42033
Jun 19, 2026 2:47:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:47:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:47:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:47:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:47:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:47:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:47:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:47:02.476200 19909 mini_ranger.cc:160] Time spent starting Ranger: real 7.984s user 0.000s sys 0.008s
Jun 19, 2026 2:47:22 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:47:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 2:47:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:47:22 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:47:23 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:47:23.802650 19909 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.113.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.113.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/rangerkms_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.113.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/rangerkms_127.19.113.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.113.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.113.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.113.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/HTTP_127.19.113.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.113.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/HTTP_127.19.113.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:47:23 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837243, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:47:23.879462 19909 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:47:23.887713 19909 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:47:24.693825 19909 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:47:24.693907 19909 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms
I20260619 02:47:24.693923 19909 mini_ranger_kms.cc:227] Using postgres at 127.19.113.84:53323
2026-06-19 02:47:24,848 [I] DB FLAVOR :POSTGRES
2026-06-19 02:47:24,848 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:47:24,848 [I] Checking connection
2026-06-19 02:47:24,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:47:25,099 [I] connection success
2026-06-19 02:47:25,099 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:47:25,099 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:47:25,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:47:25,335 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:47:25,335 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:47:25,335 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:47:25,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.113.84:53323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:47:25,604 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:47:26.062106 19909 mini_ranger_kms.cc:310] Created kms service
I20260619 02:47:26.227437 19909 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:47:26.269660 19909 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:47:26 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837243, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:26.664518 19909 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:47:26.664780 19909 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms:/tmp/dist-test-task85DHGI/build/release/bin/postgresql.jar:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task85DHGI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:47:26.664809 19909 mini_ranger_kms.cc:240] Using host: 127.19.113.84
I20260619 02:47:26.668238 19909 mini_ranger_kms.cc:276] Ranger KMS PID: 32039
I20260619 02:47:26.668308 19909 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.113.84:46971
Jun 19, 2026 2:47:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:47:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:47:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:47:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:47:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:47:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:47:28.674028 19909 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.795s user 0.003s sys 0.134s
I20260619 02:47:28.674127 19909 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:47:28.674168 19909 mini_ranger_kms.cc:401] 127.19.113.84:46971/kms/v1/keys
Jun 19 02:47:28 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837248, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.113.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:47:29 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837248, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.113.84@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
Jun 19 02:47:29 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837248, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.113.84@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
Jun 19 02:47:31 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:47:31.869814 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45055
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:45055
--ranger_config_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:46971/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:31.948885 32090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:31.949096 32090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:31.949116 32090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:31.950291 32090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:31.950335 32090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:31.950347 32090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:31.950358 32090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:31.950368 32090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:31.951911 32090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.113.84:46971/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:45055
--ranger_config_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:31.952128 32090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:31.952328 32090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:31.956781 32095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:31.956847 32096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:31.956781 32098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:31.957232 32090 server_base.cc:1034] running on GCE node
I20260619 02:47:31.957446 32090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:31.957702 32090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:31.958837 32090 hybrid_clock.cc:648] HybridClock initialized: now 1781837251958823 us; error 31 us; skew 500 ppm
Jun 19 02:47:31 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837251, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:31.964725 32090 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:47:31.965272 32090 webserver.cc:460] Webserver started at http://127.19.113.126:45875/ using document root <none> and password file <none>
I20260619 02:47:31.965572 32090 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:31.965633 32090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:31.965770 32090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:47:31 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837251, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:32.019951 32090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "ff6273c644214a08afd3d02041441977"
format_stamp: "Formatted at 2026-06-19 02:47:32 on dist-test-slave-zqpb"
server_key: "30d2a06bceff2a6add7d4b5731065be8"
server_key_iv: "4397996072e0f75cfb170c3bb28275d5"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:32.020445 32090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "ff6273c644214a08afd3d02041441977"
format_stamp: "Formatted at 2026-06-19 02:47:32 on dist-test-slave-zqpb"
server_key: "30d2a06bceff2a6add7d4b5731065be8"
server_key_iv: "4397996072e0f75cfb170c3bb28275d5"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:32.022269 32090 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:47:32.036696 32105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:32.036928 32090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:47:32.037101 32090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "ff6273c644214a08afd3d02041441977"
format_stamp: "Formatted at 2026-06-19 02:47:32 on dist-test-slave-zqpb"
server_key: "30d2a06bceff2a6add7d4b5731065be8"
server_key_iv: "4397996072e0f75cfb170c3bb28275d5"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:32.037209 32090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:32.050474 32090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:32.052246 32090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:32.052367 32090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:32.056720 32090 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:45055
I20260619 02:47:32.056747 32157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:45055 every 8 connection(s)
I20260619 02:47:32.057089 32090 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:47:32.057817 32158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:32.060806 32158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Bootstrap starting.
I20260619 02:47:32.061467 32158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:32.061762 32158 log.cc:824] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:32.062493 32158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: No bootstrap required, opened a new log
I20260619 02:47:32.065163 32158 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } }
I20260619 02:47:32.065341 32158 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:32.065372 32158 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6273c644214a08afd3d02041441977, State: Initialized, Role: FOLLOWER
I20260619 02:47:32.065526 32158 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } }
I20260619 02:47:32.065603 32158 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:47:32.065630 32158 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:47:32.065670 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 32090
I20260619 02:47:32.065699 32158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:32.065773 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:47:32.065976 19909 external_mini_cluster.cc:1428] Setting key 1af88a41e4d50040f757617d1b2c71c2
I20260619 02:47:32.066316 32158 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } }
I20260619 02:47:32.066444 32158 leader_election.cc:302] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff6273c644214a08afd3d02041441977; no voters:
I20260619 02:47:32.066660 32158 leader_election.cc:288] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:47:32.066720 32163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:47:32.066938 32163 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 1 LEADER]: Becoming Leader. State: Replica: ff6273c644214a08afd3d02041441977, State: Running, Role: LEADER
I20260619 02:47:32.066977 32158 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:32.068322 32163 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } }
I20260619 02:47:32.068886 32164 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff6273c644214a08afd3d02041441977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } } }
I20260619 02:47:32.068974 32164 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:32.069084 32164 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff6273c644214a08afd3d02041441977. Latest consensus state: current_term: 1 leader_uuid: "ff6273c644214a08afd3d02041441977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } } }
I20260619 02:47:32.069118 32164 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:32.069362 32177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:47:32.069607 32158 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:47:32.069958 32177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:47:32.071959 32177 catalog_manager.cc:1347] Generated new cluster ID: d11157fe7e7b4b2f9a2a3eda1c6dd411
I20260619 02:47:32.072036 32177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:47:32 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:32.080721 32162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:32.066417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.19.113.126:45055)
0619 02:47:32.066592 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:32.066602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:32.066637 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:47:32.066750 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:32.066758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:32.066815 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:32.067739 (+ 924us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:32.067836 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:32.069285 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:32.070419 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:32.071039 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:32.071429 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:32.073897 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:32.073902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:32.073902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:32.073914 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:32.078108 (+ 4194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:32.078570 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:32.078573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:32.078573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:32.078715 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:32.078987 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:32.078987 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:32.078988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:32.079421 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:32.080017 (+ 596us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:32.080085 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:47:32.080435 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
I20260619 02:47:32.086961 32177 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:47:32.088109 32177 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:47:32.093434 32177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Generated new TSK 0
I20260619 02:47:32.093722 32177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:47:32.769534 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:46971/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45055
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:32.886785 32204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:32.887086 32204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:32.887123 32204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:32.888928 32204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:32.888996 32204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:32.889036 32204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:47:32.891311 32204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.113.84:46971/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:32.891587 32204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:32.891870 32204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:47:32.897141 32204 server_base.cc:1034] running on GCE node
W20260619 02:47:32.897152 32209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:32.897291 32210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:32.897324 32212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:32.897657 32204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:32.897887 32204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:32.899020 32204 hybrid_clock.cc:648] HybridClock initialized: now 1781837252899007 us; error 30 us; skew 500 ppm
Jun 19 02:47:32 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:32.901499 32204 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:47:32.901917 32204 webserver.cc:460] Webserver started at http://127.19.113.65:42071/ using document root <none> and password file <none>
I20260619 02:47:32.902117 32204 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:32.902170 32204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:32.902294 32204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:47:32 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:32.914825 32204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "04d4d03f128d4489a0c106c55cb9a289"
format_stamp: "Formatted at 2026-06-19 02:47:32 on dist-test-slave-zqpb"
server_key: "3a4164f695d2351b8b7a9e95ae9e94ee"
server_key_iv: "c04b01b92fcc1868bf0db4c4fbc335f1"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:32.915267 32204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "04d4d03f128d4489a0c106c55cb9a289"
format_stamp: "Formatted at 2026-06-19 02:47:32 on dist-test-slave-zqpb"
server_key: "3a4164f695d2351b8b7a9e95ae9e94ee"
server_key_iv: "c04b01b92fcc1868bf0db4c4fbc335f1"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:32.916566 32204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:32.929513 32219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:32.929780 32204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:32.929889 32204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "04d4d03f128d4489a0c106c55cb9a289"
format_stamp: "Formatted at 2026-06-19 02:47:32 on dist-test-slave-zqpb"
server_key: "3a4164f695d2351b8b7a9e95ae9e94ee"
server_key_iv: "c04b01b92fcc1868bf0db4c4fbc335f1"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:32.929982 32204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:32.963434 32204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:32.965835 32204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:32.966004 32204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:32.966277 32204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:32.966686 32204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:32.966740 32204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:32.966771 32204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:32.966826 32204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:32.974330 32204 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:35483
I20260619 02:47:32.974390 32334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:35483 every 8 connection(s)
I20260619 02:47:32.974790 32204 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:47:32.977932 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 32204
I20260619 02:47:32.978060 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:47:32.978235 19909 external_mini_cluster.cc:1428] Setting key 106b4edcbff81f31a150b4bf84b4bec4
Jun 19 02:47:32 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:32.986414 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:32.975610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45055 (local address 127.19.113.65:47163)
0619 02:47:32.975778 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:32.975826 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:32.976405 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:32.976677 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:32.976680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:32.976897 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:32.977102 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:32.977106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:32.978467 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:32.978468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:32.979660 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:32.979663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:32.980241 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:32.980242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:32.980464 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:32.981053 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:32.981065 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:32.982753 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:32.984746 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:32.984753 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:32.984755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:32.985049 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:32.985266 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:32.985267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:32.985268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:32.985394 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:32.985693 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:32.985699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:32.985873 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 02:47:32.986002 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 02:47:32.986164 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260619 02:47:32.986846 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:32.975617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:47163 (local address 127.19.113.126:45055)
0619 02:47:32.975746 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:32.975748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:32.975928 (+ 180us) server_negotiation.cc:407] Connection header received
0619 02:47:32.976520 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:32.976522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:32.976536 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:32.976624 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:32.977189 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:32.977724 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:32.979757 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:32.980078 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:32.980156 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:32.982864 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:32.982889 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:32.982890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:32.982898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:32.984644 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:32.985133 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:32.985134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:32.985134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:32.985199 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:32.985466 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:32.985467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:32.985467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:32.985603 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:32.985666 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:32.986667 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0619 02:47:32.986750 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
I20260619 02:47:32.987334 32335 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45055
I20260619 02:47:32.987437 32335 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:32.987630 32335 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:32.988183 32122 ts_manager.cc:194] Registered new tserver with Master: 04d4d03f128d4489a0c106c55cb9a289 (127.19.113.65:35483)
I20260619 02:47:32.989107 32122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:47163
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:47:33.007109 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:46971/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45055
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 02:47:33.121557 32341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:33.121824 32341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:33.121857 32341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:33.123646 32341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:33.123705 32341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:33.123746 32341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:47:33.125949 32341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.113.84:46971/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:33.126199 32341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:33.126474 32341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:33.133289 32349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:33.133610 32348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:33.133803 32351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:33.134652 32341 server_base.cc:1034] running on GCE node
I20260619 02:47:33.134927 32341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:33.135227 32341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:33.136396 32341 hybrid_clock.cc:648] HybridClock initialized: now 1781837253136373 us; error 34 us; skew 500 ppm
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:33.139638 32341 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:47:33.140187 32341 webserver.cc:460] Webserver started at http://127.19.113.66:33687/ using document root <none> and password file <none>
I20260619 02:47:33.140472 32341 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:33.140537 32341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:33.140668 32341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:33.158910 32341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "a71ca12d6c484662aac97beda91ec3cb"
format_stamp: "Formatted at 2026-06-19 02:47:33 on dist-test-slave-zqpb"
server_key: "307281686e719148570d7eb69198284a"
server_key_iv: "dea98301ff382ec7f4dd43a5bc2eb75a"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:33.159492 32341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "a71ca12d6c484662aac97beda91ec3cb"
format_stamp: "Formatted at 2026-06-19 02:47:33 on dist-test-slave-zqpb"
server_key: "307281686e719148570d7eb69198284a"
server_key_iv: "dea98301ff382ec7f4dd43a5bc2eb75a"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:33.163296 32341 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.001s
I20260619 02:47:33.184497 32358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.184729 32341 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.001s
I20260619 02:47:33.184823 32341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "a71ca12d6c484662aac97beda91ec3cb"
format_stamp: "Formatted at 2026-06-19 02:47:33 on dist-test-slave-zqpb"
server_key: "307281686e719148570d7eb69198284a"
server_key_iv: "dea98301ff382ec7f4dd43a5bc2eb75a"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:33.184933 32341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:33.218676 32341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:33.220618 32341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:33.220723 32341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:33.220937 32341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:33.221288 32341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:33.221339 32341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.221376 32341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:33.221411 32341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.227984 32341 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:40887
I20260619 02:47:33.228149 32472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:40887 every 8 connection(s)
I20260619 02:47:33.228368 32341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:33.236447 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 32341
I20260619 02:47:33.236544 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:47:33.236756 19909 external_mini_cluster.cc:1428] Setting key 1a58ab42445bbb627d27549cbbb20260
I20260619 02:47:33.237208 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:33.229154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:51369 (local address 127.19.113.126:45055)
0619 02:47:33.229196 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:33.229198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:33.229449 (+ 251us) server_negotiation.cc:407] Connection header received
0619 02:47:33.229938 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:33.229939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:33.229955 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:33.230014 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:33.230544 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.231118 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.232165 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.232382 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.232451 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:33.234368 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:33.234369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:33.234370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:33.234377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:33.235736 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:33.236108 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:33.236110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:33.236110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:33.236165 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:33.236484 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:33.236485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:33.236486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:33.236629 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:33.236676 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:33.237051 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 02:47:33.237139 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:47:33.237978 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:33.229168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45055 (local address 127.19.113.66:51369)
0619 02:47:33.229323 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:33.229360 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:33.229845 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:33.230082 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:33.230085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:33.230266 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:33.230456 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:33.230460 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.231203 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.231204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:33.232082 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:33.232085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.232463 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.232464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:33.232577 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:33.233066 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:33.233075 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:33.234282 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:33.235815 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:33.235822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:33.235824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:33.236041 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:33.236216 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:33.236216 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:33.236217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:33.236351 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:33.236725 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:33.236732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:33.236894 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 02:47:33.237581 (+ 687us) client_negotiation.cc:240] Negotiation successful
0619 02:47:33.237750 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260619 02:47:33.238444 32473 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45055
I20260619 02:47:33.238540 32473 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:33.238735 32473 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:33.239143 32122 ts_manager.cc:194] Registered new tserver with Master: a71ca12d6c484662aac97beda91ec3cb (127.19.113.66:40887)
I20260619 02:47:33.239689 32122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:51369
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:47:33.263233 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:46971/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45055
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:33.354128 32478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:33.354491 32478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:33.354588 32478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:33.356274 32478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:33.356324 32478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:33.356362 32478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:47:33.357985 32478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.113.84:46971/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:33.358209 32478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:33.358487 32478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:33.363756 32486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:33.363756 32483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:33.364377 32478 server_base.cc:1034] running on GCE node
W20260619 02:47:33.366493 32484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:33.366716 32478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:33.367060 32478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:33.368209 32478 hybrid_clock.cc:648] HybridClock initialized: now 1781837253368200 us; error 31 us; skew 500 ppm
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:33.371280 32478 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:47:33.371831 32478 webserver.cc:460] Webserver started at http://127.19.113.67:37973/ using document root <none> and password file <none>
I20260619 02:47:33.372053 32478 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:33.372088 32478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:33.372177 32478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:33.386282 32478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec"
format_stamp: "Formatted at 2026-06-19 02:47:33 on dist-test-slave-zqpb"
server_key: "b8f057112b2943715f73441fe4165536"
server_key_iv: "02fcd6a59b16522f7e738bf54c6d080b"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:33.386737 32478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec"
format_stamp: "Formatted at 2026-06-19 02:47:33 on dist-test-slave-zqpb"
server_key: "b8f057112b2943715f73441fe4165536"
server_key_iv: "02fcd6a59b16522f7e738bf54c6d080b"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:33.388636 32478 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260619 02:47:33.401909 32493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.402297 32478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:33.402431 32478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec"
format_stamp: "Formatted at 2026-06-19 02:47:33 on dist-test-slave-zqpb"
server_key: "b8f057112b2943715f73441fe4165536"
server_key_iv: "02fcd6a59b16522f7e738bf54c6d080b"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:33.402576 32478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:33.415601 32478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:33.417904 32478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:33.418068 32478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:33.418334 32478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:33.418671 32478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:33.418706 32478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.418735 32478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:33.418751 32478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.425297 32478 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:37193
I20260619 02:47:33.425360 32606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:37193 every 8 connection(s)
I20260619 02:47:33.425678 32478 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:47:33.428619 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 32478
I20260619 02:47:33.428731 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:47:33.428921 19909 external_mini_cluster.cc:1428] Setting key 92da7d3b0103695b75596e35ce3c7f1c
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:33.437527 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:33.426681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:49427 (local address 127.19.113.126:45055)
0619 02:47:33.426725 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:33.426727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:33.426984 (+ 257us) server_negotiation.cc:407] Connection header received
0619 02:47:33.427665 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:33.427667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:33.427678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:33.427708 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:33.429599 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.430369 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.431931 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.432247 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.432293 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:33.434985 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:33.434986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:33.434987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:33.434994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:33.436258 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:33.436658 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:33.436660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:33.436660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:33.436708 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:33.436925 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:33.436926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:33.436927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:33.437036 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:33.437087 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:33.437352 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 02:47:33.437432 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:47:33.437680 32609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:33.426386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45055 (local address 127.19.113.67:49427)
0619 02:47:33.426812 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:33.426858 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:33.427538 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:33.427939 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:33.427943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:33.428181 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:33.429316 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:33.429323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.430513 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.430514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:33.431781 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:33.431784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.432395 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.432396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:33.432529 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:33.433129 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:33.433139 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:33.434892 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:33.436333 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:33.436340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:33.436342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:33.436579 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:33.436758 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:33.436759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:33.436759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:33.436864 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:33.437088 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:33.437094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:33.437227 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 02:47:33.437308 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:47:33.437449 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":91,"threads_started":1}
I20260619 02:47:33.438321 32607 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45055
I20260619 02:47:33.438416 32607 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:33.438596 32607 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:33.438983 32122 ts_manager.cc:194] Registered new tserver with Master: c1d9dadda26f4d6d9fb01dca2109e4ec (127.19.113.67:37193)
I20260619 02:47:33.439334 32122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:49427
I20260619 02:47:33.441200 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:47:33.441807 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 32204
I20260619 02:47:33.446836 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 32341
I20260619 02:47:33.450598 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 32478
I20260619 02:47:33.455662 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 32090
I20260619 02:47:33.460276 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45055
--webserver_interface=127.19.113.126
--webserver_port=45875
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:45055
--ranger_config_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:46971/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.113.84:46971/kms with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:33.539685 32610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:33.539898 32610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:33.539929 32610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:33.541078 32610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:33.541119 32610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:33.541137 32610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:33.541152 32610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:33.541167 32610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:33.542766 32610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.113.84:46971/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:45055
--ranger_config_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=45875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:33.543066 32610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:33.543273 32610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:33.548092 32615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:33.548130 32618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:33.548177 32610 server_base.cc:1034] running on GCE node
W20260619 02:47:33.548101 32616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:33.548540 32610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:33.548789 32610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:33.549933 32610 hybrid_clock.cc:648] HybridClock initialized: now 1781837253549897 us; error 49 us; skew 500 ppm
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:33.552372 32610 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:47:33.552801 32610 webserver.cc:460] Webserver started at http://127.19.113.126:45875/ using document root <none> and password file <none>
I20260619 02:47:33.553002 32610 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:33.553054 32610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:33.628432 32610 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:33.629290 32626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.629489 32610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:33.629601 32610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "ff6273c644214a08afd3d02041441977"
format_stamp: "Formatted at 2026-06-19 02:47:32 on dist-test-slave-zqpb"
server_key: "30d2a06bceff2a6add7d4b5731065be8"
server_key_iv: "4397996072e0f75cfb170c3bb28275d5"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:33.630070 32610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:33.660692 32610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:33.662312 32610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:33.662432 32610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:33.666304 32610 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:45055
I20260619 02:47:33.666370 32678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:45055 every 8 connection(s)
I20260619 02:47:33.666714 32610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:47:33.668241 32679 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:47:33.669137 32679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Bootstrap starting.
I20260619 02:47:33.676426 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 32610
I20260619 02:47:33.676764 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:46971/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:35483
--local_ip_for_outbound_sockets=127.19.113.65
--tserver_master_addrs=127.19.113.126:45055
--webserver_port=42071
--webserver_interface=127.19.113.65
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.113.84:46971/kms with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:47:33.678951 32679 log.cc:824] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:33.681226 32679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:47:33.681555 32679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Bootstrap complete.
I20260619 02:47:33.684958 32679 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } }
I20260619 02:47:33.685305 32679 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6273c644214a08afd3d02041441977, State: Initialized, Role: FOLLOWER
I20260619 02:47:33.685472 32679 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } }
I20260619 02:47:33.685547 32679 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:47:33.685593 32679 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:47:33.685639 32679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:47:33.687009 32679 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } }
I20260619 02:47:33.687121 32679 leader_election.cc:302] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff6273c644214a08afd3d02041441977; no voters:
I20260619 02:47:33.687288 32679 leader_election.cc:288] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:47:33.687394 32683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:47:33.687547 32679 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:33.687566 32683 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [term 2 LEADER]: Becoming Leader. State: Replica: ff6273c644214a08afd3d02041441977, State: Running, Role: LEADER
I20260619 02:47:33.687745 32683 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } }
I20260619 02:47:33.688047 32683 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff6273c644214a08afd3d02041441977. Latest consensus state: current_term: 2 leader_uuid: "ff6273c644214a08afd3d02041441977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } } }
I20260619 02:47:33.688072 32684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ff6273c644214a08afd3d02041441977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6273c644214a08afd3d02041441977" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45055 } } }
I20260619 02:47:33.688182 32683 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:33.688202 32684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977 [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:33.688474 32692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:47:33.688983 32692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:47:33.689162 32692 catalog_manager.cc:1259] Loaded cluster ID: d11157fe7e7b4b2f9a2a3eda1c6dd411
I20260619 02:47:33.689229 32692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:47:33.690618 32692 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:47:33.690843 32692 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Loaded TSK: 0
I20260619 02:47:33.691162 32692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:47:33.695307 32679 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260619 02:47:33.779374 32681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:33.779587 32681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:33.779618 32681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:33.780803 32681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:33.780845 32681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:33.780880 32681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:47:33.782727 32681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.113.84:46971/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:35483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=42071
--tserver_master_addrs=127.19.113.126:45055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:33.782990 32681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:33.783236 32681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:33.789500 32718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:33.789424 32716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:33.789569 32715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:33.789955 32681 server_base.cc:1034] running on GCE node
I20260619 02:47:33.790341 32681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:33.790647 32681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:33.791831 32681 hybrid_clock.cc:648] HybridClock initialized: now 1781837253791819 us; error 26 us; skew 500 ppm
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:33.794195 32681 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:47:33.794705 32681 webserver.cc:460] Webserver started at http://127.19.113.65:42071/ using document root <none> and password file <none>
I20260619 02:47:33.794976 32681 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:33.795056 32681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:33.816712 32681 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:47:33.817814 32727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.817988 32681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:33.818080 32681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "04d4d03f128d4489a0c106c55cb9a289"
format_stamp: "Formatted at 2026-06-19 02:47:32 on dist-test-slave-zqpb"
server_key: "3a4164f695d2351b8b7a9e95ae9e94ee"
server_key_iv: "c04b01b92fcc1868bf0db4c4fbc335f1"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:33.818571 32681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:33.839955 32681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:33.842284 32681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:33.842450 32681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:33.842713 32681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:33.843096 32681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:33.843142 32681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.843169 32681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:33.843191 32681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:33.849531 32681 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:35483
I20260619 02:47:33.849933 32681 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:47:33.852018 372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:35483 every 8 connection(s)
I20260619 02:47:33.854725 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 32681
I20260619 02:47:33.855026 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:46971/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:40887
--local_ip_for_outbound_sockets=127.19.113.66
--tserver_master_addrs=127.19.113.126:45055
--webserver_port=33687
--webserver_interface=127.19.113.66
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.113.84:46971/kms with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:33.866617 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:33.852713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45055 (local address 127.19.113.65:59403)
0619 02:47:33.852969 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:33.853009 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:33.853599 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:33.855718 (+ 2119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:33.855723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:33.855966 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:33.856253 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:33.856260 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.858003 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.858004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:33.859294 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:33.859298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.860209 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.860210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:33.860371 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:33.860891 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:33.860899 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:33.862153 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:33.864799 (+ 2646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:33.864812 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:33.864815 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:33.865116 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:33.865403 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:33.865404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:33.865404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:33.865550 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:33.866022 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:33.866028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:33.866182 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 02:47:33.866250 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:47:33.866367 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":82,"threads_started":1}
I20260619 02:47:33.867339 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:33.852800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:59403 (local address 127.19.113.126:45055)
0619 02:47:33.855009 (+ 2209us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:33.855018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:33.855266 (+ 248us) server_negotiation.cc:407] Connection header received
0619 02:47:33.855430 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:33.855437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:33.855511 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:33.856392 (+ 881us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:33.856518 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.857884 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.859447 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:33.860085 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:33.860193 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:33.862726 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:33.862729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:33.862730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:33.862740 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:33.864698 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:33.865217 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:33.865220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:33.865221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:33.865328 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:33.865629 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:33.865630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:33.865630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:33.865936 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:33.866951 (+ 1015us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:33.867023 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 02:47:33.867144 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2146,"thread_start_us":75,"threads_started":1}
I20260619 02:47:33.868021 373 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45055
I20260619 02:47:33.868124 373 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:33.868311 373 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
W20260619 02:47:33.869107 373 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:45055 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:47:33.869169 373 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:33.869232 373 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:33.869537 373 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:33.869612 373 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
W20260619 02:47:33.869874 373 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:47:33.978169 377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:33.978475 377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:33.978516 377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:33.980281 377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:33.980351 377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:33.980397 377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:47:33.982628 377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.113.84:46971/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:40887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=33687
--tserver_master_addrs=127.19.113.126:45055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:33.982925 377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:33.983225 377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:33.988927 383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:33.988916 382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:33.989442 385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:33.989784 377 server_base.cc:1034] running on GCE node
I20260619 02:47:33.990021 377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:33.990343 377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:33.991514 377 hybrid_clock.cc:648] HybridClock initialized: now 1781837253991434 us; error 91 us; skew 500 ppm
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:33.994227 377 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:47:33.994781 377 webserver.cc:460] Webserver started at http://127.19.113.66:33687/ using document root <none> and password file <none>
I20260619 02:47:33.995074 377 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:33.995159 377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:47:33 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:34.018790 377 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:34.019852 393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:34.020013 377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:47:34.020123 377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "a71ca12d6c484662aac97beda91ec3cb"
format_stamp: "Formatted at 2026-06-19 02:47:33 on dist-test-slave-zqpb"
server_key: "307281686e719148570d7eb69198284a"
server_key_iv: "dea98301ff382ec7f4dd43a5bc2eb75a"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:34.020596 377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:34.035398 377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:34.037294 377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:34.037420 377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:34.037642 377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:34.037990 377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:34.038028 377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:34.038058 377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:34.038092 377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:34.044729 377 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:40887
I20260619 02:47:34.045181 377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:47:34.045642 506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:40887 every 8 connection(s)
I20260619 02:47:34.052182 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 377
I20260619 02:47:34.052470 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.113.84:46971/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:37193
--local_ip_for_outbound_sockets=127.19.113.67
--tserver_master_addrs=127.19.113.126:45055
--webserver_port=37973
--webserver_interface=127.19.113.67
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.113.84:46971/kms with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:47:34 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837253, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:34.056443 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:34.046623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:52209 (local address 127.19.113.126:45055)
0619 02:47:34.046671 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:34.046673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:34.046685 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:47:34.047363 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:34.047365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:34.047381 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:34.047450 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:34.048104 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.048857 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.050340 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.050631 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.050712 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:34.053172 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:34.053174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:34.053175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:34.053182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:34.054843 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:34.055373 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:34.055375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:34.055375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:34.055436 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:34.055702 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:34.055703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:34.055703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:34.055841 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:34.055905 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:34.056246 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 02:47:34.056337 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:47:34.056602 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:34.046340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45055 (local address 127.19.113.66:52209)
0619 02:47:34.046518 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:34.046565 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:34.047238 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:34.047516 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:34.047521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:34.047737 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:34.047996 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:34.048001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.048975 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.048977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:34.050239 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:34.050242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.050751 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.050752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:34.050920 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:34.051476 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:34.051486 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:34.053047 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:34.054960 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:34.054968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:34.054970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:34.055286 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:34.055498 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:34.055499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:34.055499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:34.055629 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:34.055901 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:34.055908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:34.056087 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 02:47:34.056213 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 02:47:34.056371 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260619 02:47:34.057096 507 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45055
I20260619 02:47:34.057204 507 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:34.057403 507 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
W20260619 02:47:34.058113 507 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:45055 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:47:34.058190 507 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:34.058249 507 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:34.058571 507 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:34.058651 507 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
W20260619 02:47:34.058983 507 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:47:34.169567 510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:34.169772 510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:34.169802 510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:34.170981 510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:34.171020 510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:34.171054 510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:47:34.172791 510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.113.84:46971/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:37193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=37973
--tserver_master_addrs=127.19.113.126:45055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:34.173031 510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:34.173255 510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:34.178184 515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:34.178184 518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:34.178504 516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:34.179105 510 server_base.cc:1034] running on GCE node
I20260619 02:47:34.179319 510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:34.179600 510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:34.180778 510 hybrid_clock.cc:648] HybridClock initialized: now 1781837254180757 us; error 32 us; skew 500 ppm
Jun 19 02:47:34 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:34.183987 510 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:47:34.184536 510 webserver.cc:460] Webserver started at http://127.19.113.67:37973/ using document root <none> and password file <none>
I20260619 02:47:34.184803 510 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:34.184888 510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:47:34 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:34.211136 510 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260619 02:47:34.212177 527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:34.212320 510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:34.212391 510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec"
format_stamp: "Formatted at 2026-06-19 02:47:33 on dist-test-slave-zqpb"
server_key: "b8f057112b2943715f73441fe4165536"
server_key_iv: "02fcd6a59b16522f7e738bf54c6d080b"
server_key_version: "kuduclusterkey@0"
I20260619 02:47:34.212831 510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:34.228622 510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:34.230459 510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:34.230607 510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:34.230795 510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:34.231145 510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:34.231180 510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:34.231209 510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:34.231232 510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:34.236505 510 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:37193
I20260619 02:47:34.236868 510 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:47:34.237119 640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:37193 every 8 connection(s)
Jun 19 02:47:34 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:34.244771 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 510
I20260619 02:47:34.248654 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:34.237842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45055 (local address 127.19.113.67:33357)
0619 02:47:34.238843 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:34.238913 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:34.239527 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:34.239817 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:34.239822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:34.240060 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:34.240331 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:34.240337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.241344 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.241345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:34.242619 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:34.242622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.243131 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.243132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:34.243241 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:34.243712 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:34.243720 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:34.245250 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:34.246962 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:34.246968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:34.246971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:34.247285 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:34.247508 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:34.247508 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:34.247509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:34.247648 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:34.247957 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:34.247964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:34.248142 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 02:47:34.248257 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 02:47:34.248417 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"thread_start_us":79,"threads_started":1}
I20260619 02:47:34.248992 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:34.237682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:33357 (local address 127.19.113.126:45055)
0619 02:47:34.237728 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:34.237730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:34.239007 (+ 1277us) server_negotiation.cc:407] Connection header received
0619 02:47:34.239647 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:34.239649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:34.239662 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:34.239724 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:34.240434 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.241247 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.242707 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.243012 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.243068 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:34.245350 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:34.245351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:34.245351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:34.245358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:34.246844 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:34.247375 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:34.247376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:34.247376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:34.247438 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:34.247728 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:34.247729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:34.247729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:34.247865 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:34.247928 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:34.248804 (+ 876us) server_negotiation.cc:299] Negotiation successful
0619 02:47:34.248906 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:47:34.251816 641 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45055
I20260619 02:47:34.251920 641 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:34.252138 641 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
W20260619 02:47:34.252823 641 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:45055 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:47:34.252895 641 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:34.252975 641 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:34.253288 641 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:34.253360 641 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:34.254243 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:34.245279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.19.113.126:45055)
0619 02:47:34.246039 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:34.246041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:34.246048 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:47:34.246066 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:34.246067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:34.246078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:34.246817 (+ 739us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:34.246853 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.247656 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.249218 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:34.249544 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:34.249605 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:34.250991 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:34.250993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:34.250993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:34.251000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:34.252951 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:34.253517 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:34.253518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:34.253519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:34.253567 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:34.253746 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:34.253747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:34.253747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:34.253859 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:34.254032 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:34.254068 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 02:47:34.254145 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":55,"threads_started":1}
W20260619 02:47:34.255118 641 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 02:47:34 dist-test-slave-zqpb krb5kdc[31646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:34.870076 373 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:34.870249 373 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:34.870713 32643 ts_manager.cc:194] Registered new tserver with Master: 04d4d03f128d4489a0c106c55cb9a289 (127.19.113.65:35483)
I20260619 02:47:34.871742 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:59403
I20260619 02:47:35.059173 507 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:35.059311 507 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:35.059638 32643 ts_manager.cc:194] Registered new tserver with Master: a71ca12d6c484662aac97beda91ec3cb (127.19.113.66:40887)
I20260619 02:47:35.060135 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:52209
Jun 19 02:47:35 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:35.255277 641 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:35.255437 641 heartbeater.cc:511] Master 127.19.113.126:45055 requested a full tablet report, sending...
I20260619 02:47:35.255750 32643 ts_manager.cc:194] Registered new tserver with Master: c1d9dadda26f4d6d9fb01dca2109e4ec (127.19.113.67:37193)
I20260619 02:47:35.256265 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:33357
I20260619 02:47:35.266413 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:47:35.275941 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.267910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.19.113.126:45055)
0619 02:47:35.268617 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:35.268619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:35.268727 (+ 108us) server_negotiation.cc:407] Connection header received
0619 02:47:35.268795 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:35.268796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:35.268807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:35.268896 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:35.269519 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.270300 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.271361 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.271684 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.271758 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.273108 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:35.273110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:35.273110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:35.273117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:35.274777 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:35.275164 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:35.275165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:35.275166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:35.275231 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:35.275421 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:35.275423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:35.275423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:35.275560 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:35.275664 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:35.275761 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 02:47:35.275843 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":86,"threads_started":1}
I20260619 02:47:35.278241 32643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:47:35.280676 32643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:47:35.291548 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.283774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:37193 (local address 127.0.0.1:54030)
0619 02:47:35.287143 (+ 3369us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:35.287150 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:35.287185 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:35.287442 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:35.287442 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:35.287445 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:35.287505 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.287507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.289057 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.289057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.290215 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.290217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.291238 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.291239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.291328 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.291342 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:47:35.291415 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:47:35.291463 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3356,"thread_start_us":59,"threads_started":1}
I20260619 02:47:35.291942 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.284091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.19.113.67:37193)
0619 02:47:35.284533 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:35.284538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:35.287233 (+ 2695us) server_negotiation.cc:407] Connection header received
0619 02:47:35.287270 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:35.287273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:35.287288 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:35.287349 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:35.287571 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.288977 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.290305 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.291080 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.291144 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.291199 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:35.291824 (+ 625us) server_negotiation.cc:299] Negotiation successful
0619 02:47:35.291867 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":73,"threads_started":1}
I20260619 02:47:35.292105 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.283803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.19.113.66:40887)
0619 02:47:35.284009 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:35.284013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:35.284251 (+ 238us) server_negotiation.cc:407] Connection header received
0619 02:47:35.284286 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:35.284289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:35.284304 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:35.284364 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:35.284591 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.285649 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.290660 (+ 5011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.291693 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.291745 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.291794 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:35.291984 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:47:35.292026 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":47,"threads_started":1}
I20260619 02:47:35.292714 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.283545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:40887 (local address 127.0.0.1:50746)
0619 02:47:35.284077 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:35.284114 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:35.284179 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:35.284459 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:35.284461 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:35.284463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:35.284524 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.284525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.285732 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.285733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.290566 (+ 4833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.290568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.291847 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.291848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.291909 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.291912 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:47:35.292603 (+ 691us) client_negotiation.cc:240] Negotiation successful
0619 02:47:35.292650 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":91,"threads_started":1}
I20260619 02:47:35.292846 575 tablet_service.cc:1467] Processing CreateTablet for tablet 72183254e4d547b085da5c365b224252 (DEFAULT_TABLE table=test-table [id=0de983d3bf484be88b7d8d7639deac18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:35.293203 575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72183254e4d547b085da5c365b224252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:35.293702 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.283548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:35483 (local address 127.0.0.1:42564)
0619 02:47:35.286970 (+ 3422us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:35.286981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:35.287031 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:35.287373 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:35.287374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:35.287376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:35.287425 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.287426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.288711 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.288712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.289761 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.289764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.293502 (+ 3738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.293503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.293563 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.293566 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:47:35.293607 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 02:47:35.293641 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3404,"thread_start_us":89,"threads_started":1}
I20260619 02:47:35.293937 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.283960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42564 (local address 127.19.113.65:35483)
0619 02:47:35.284609 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:35.284613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:35.287142 (+ 2529us) server_negotiation.cc:407] Connection header received
0619 02:47:35.287182 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:35.287185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:35.287202 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:35.287280 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:35.287500 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.288476 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.289851 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.293110 (+ 3259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.293180 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.293229 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:35.293821 (+ 592us) server_negotiation.cc:299] Negotiation successful
0619 02:47:35.293866 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":46,"threads_started":1}
I20260619 02:47:35.293850 441 tablet_service.cc:1467] Processing CreateTablet for tablet 72183254e4d547b085da5c365b224252 (DEFAULT_TABLE table=test-table [id=0de983d3bf484be88b7d8d7639deac18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:35.294171 441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72183254e4d547b085da5c365b224252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:35.294411 307 tablet_service.cc:1467] Processing CreateTablet for tablet 72183254e4d547b085da5c365b224252 (DEFAULT_TABLE table=test-table [id=0de983d3bf484be88b7d8d7639deac18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:35.294667 307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72183254e4d547b085da5c365b224252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:35.297151 674 tablet_bootstrap.cc:492] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Bootstrap starting.
I20260619 02:47:35.297817 674 tablet_bootstrap.cc:654] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:35.298156 674 log.cc:824] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:35.298491 676 tablet_bootstrap.cc:492] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289: Bootstrap starting.
I20260619 02:47:35.299072 676 tablet_bootstrap.cc:654] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:35.299347 676 log.cc:824] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:35.300134 674 tablet_bootstrap.cc:492] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: No bootstrap required, opened a new log
I20260619 02:47:35.300246 676 tablet_bootstrap.cc:492] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289: No bootstrap required, opened a new log
I20260619 02:47:35.300247 674 ts_tablet_manager.cc:1397] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:47:35.300375 676 ts_tablet_manager.cc:1397] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:47:35.302619 674 raft_consensus.cc:348] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.302743 676 raft_consensus.cc:348] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.302902 674 raft_consensus.cc:374] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:35.302977 676 raft_consensus.cc:374] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:35.303010 674 raft_consensus.cc:729] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d9dadda26f4d6d9fb01dca2109e4ec, State: Initialized, Role: FOLLOWER
I20260619 02:47:35.303084 676 raft_consensus.cc:729] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d4d03f128d4489a0c106c55cb9a289, State: Initialized, Role: FOLLOWER
I20260619 02:47:35.303174 674 consensus_queue.cc:260] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.303261 676 consensus_queue.cc:260] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.303535 674 ts_tablet_manager.cc:1428] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:47:35.303562 676 ts_tablet_manager.cc:1428] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:47:35.303668 373 heartbeater.cc:503] Master 127.19.113.126:45055 was elected leader, sending a full tablet report...
I20260619 02:47:35.303704 641 heartbeater.cc:503] Master 127.19.113.126:45055 was elected leader, sending a full tablet report...
I20260619 02:47:35.307008 675 tablet_bootstrap.cc:492] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: Bootstrap starting.
I20260619 02:47:35.307529 675 tablet_bootstrap.cc:654] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:35.307777 675 log.cc:824] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:35.308539 675 tablet_bootstrap.cc:492] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: No bootstrap required, opened a new log
I20260619 02:47:35.308611 675 ts_tablet_manager.cc:1397] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:47:35.310686 675 raft_consensus.cc:348] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.310935 675 raft_consensus.cc:374] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:35.310993 675 raft_consensus.cc:729] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a71ca12d6c484662aac97beda91ec3cb, State: Initialized, Role: FOLLOWER
I20260619 02:47:35.311120 675 consensus_queue.cc:260] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.311362 675 ts_tablet_manager.cc:1428] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:47:35.311409 507 heartbeater.cc:503] Master 127.19.113.126:45055 was elected leader, sending a full tablet report...
W20260619 02:47:35.353425 374 tablet.cc:2369] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 02:47:35 dist-test-slave-zqpb krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for HTTP/127.19.113.84@KRBTEST.COM
I20260619 02:47:35.477900 679 raft_consensus.cc:482] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:35.478051 679 raft_consensus.cc:504] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.478488 679 leader_election.cc:288] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04d4d03f128d4489a0c106c55cb9a289 (127.19.113.65:35483), a71ca12d6c484662aac97beda91ec3cb (127.19.113.66:40887)
I20260619 02:47:35.482692 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.478937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:56429 (local address 127.19.113.66:40887)
0619 02:47:35.478981 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:35.478983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:35.479209 (+ 226us) server_negotiation.cc:407] Connection header received
0619 02:47:35.479304 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:35.479306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:35.479315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:35.479341 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:35.479546 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.480239 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.481645 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.482241 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.482319 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.482329 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:35.482567 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 02:47:35.482617 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:47:35.482957 685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.478650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:40887 (local address 127.19.113.67:56429)
0619 02:47:35.479148 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:35.479158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:35.479238 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:35.479414 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:35.479415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:35.479416 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:35.479486 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.479488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.480332 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.480333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.481541 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.481546 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.482403 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.482404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.482482 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.482487 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:47:35.482790 (+ 303us) client_negotiation.cc:240] Negotiation successful
0619 02:47:35.482839 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":114,"threads_started":1}
I20260619 02:47:35.483619 461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72183254e4d547b085da5c365b224252" candidate_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a71ca12d6c484662aac97beda91ec3cb" is_pre_election: true
I20260619 02:47:35.483784 461 raft_consensus.cc:2454] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d9dadda26f4d6d9fb01dca2109e4ec in term 0.
I20260619 02:47:35.484035 528 leader_election.cc:302] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a71ca12d6c484662aac97beda91ec3cb, c1d9dadda26f4d6d9fb01dca2109e4ec; no voters:
I20260619 02:47:35.484185 679 raft_consensus.cc:2781] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:47:35.484254 679 raft_consensus.cc:482] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:47:35.484274 679 raft_consensus.cc:3037] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:35.485112 679 raft_consensus.cc:504] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.485234 679 leader_election.cc:288] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [CANDIDATE]: Term 1 election: Requested vote from peers 04d4d03f128d4489a0c106c55cb9a289 (127.19.113.65:35483), a71ca12d6c484662aac97beda91ec3cb (127.19.113.66:40887)
I20260619 02:47:35.485535 461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72183254e4d547b085da5c365b224252" candidate_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a71ca12d6c484662aac97beda91ec3cb"
I20260619 02:47:35.485625 461 raft_consensus.cc:3037] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:35.485785 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.478944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:48813 (local address 127.19.113.65:35483)
0619 02:47:35.478985 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:35.478987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:35.481710 (+ 2723us) server_negotiation.cc:407] Connection header received
0619 02:47:35.481797 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:35.481798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:35.481807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:35.481831 (+ 24us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:35.482000 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.482668 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.484116 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.484656 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.484725 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.484733 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:35.485654 (+ 921us) server_negotiation.cc:299] Negotiation successful
0619 02:47:35.485700 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:47:35.486114 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.478688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:35483 (local address 127.19.113.67:48813)
0619 02:47:35.481660 (+ 2972us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:35.481670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:35.481736 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:35.481883 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:35.481884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:35.481885 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:35.481939 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.481940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.482958 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.482959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.484007 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:35.484010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.485476 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.485477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:35.485546 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.485549 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:47:35.486002 (+ 453us) client_negotiation.cc:240] Negotiation successful
0619 02:47:35.486044 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2956,"thread_start_us":94,"threads_started":1}
I20260619 02:47:35.486440 461 raft_consensus.cc:2454] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d9dadda26f4d6d9fb01dca2109e4ec in term 1.
I20260619 02:47:35.486645 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72183254e4d547b085da5c365b224252" candidate_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d4d03f128d4489a0c106c55cb9a289" is_pre_election: true
I20260619 02:47:35.486778 326 raft_consensus.cc:2454] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d9dadda26f4d6d9fb01dca2109e4ec in term 0.
I20260619 02:47:35.486897 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72183254e4d547b085da5c365b224252" candidate_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d4d03f128d4489a0c106c55cb9a289"
I20260619 02:47:35.486964 327 raft_consensus.cc:3037] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:35.487740 327 raft_consensus.cc:2454] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d9dadda26f4d6d9fb01dca2109e4ec in term 1.
I20260619 02:47:35.487910 528 leader_election.cc:302] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a71ca12d6c484662aac97beda91ec3cb, c1d9dadda26f4d6d9fb01dca2109e4ec; no voters:
I20260619 02:47:35.488085 679 raft_consensus.cc:2781] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:47:35.488237 679 raft_consensus.cc:686] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 1 LEADER]: Becoming Leader. State: Replica: c1d9dadda26f4d6d9fb01dca2109e4ec, State: Running, Role: LEADER
I20260619 02:47:35.488319 679 consensus_queue.cc:237] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } }
I20260619 02:47:35.489209 32643 catalog_manager.cc:5495] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec reported cstate change: term changed from 0 to 1, leader changed from <none> to c1d9dadda26f4d6d9fb01dca2109e4ec (127.19.113.67). New cstate: current_term: 1 leader_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1d9dadda26f4d6d9fb01dca2109e4ec" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 37193 } health_report { overall_health: HEALTHY } } }
W20260619 02:47:35.489569 642 tablet.cc:2369] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:47:35.513777 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.509841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.19.113.67:37193)
0619 02:47:35.510411 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:35.510413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:35.510420 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:47:35.510461 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:35.510461 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:35.510484 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:35.510553 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:47:35.510760 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.511520 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.512600 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:35.512895 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:35.512963 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:35.513293 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:47:35.513436 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:47:35.513508 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:35.513674 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 02:47:35.513695 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533}
I20260619 02:47:35.517052 461 raft_consensus.cc:1261] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 1 FOLLOWER]: Refusing update from remote peer c1d9dadda26f4d6d9fb01dca2109e4ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:47:35.517216 327 raft_consensus.cc:1261] T 72183254e4d547b085da5c365b224252 P 04d4d03f128d4489a0c106c55cb9a289 [term 1 FOLLOWER]: Refusing update from remote peer c1d9dadda26f4d6d9fb01dca2109e4ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:47:35.517504 679 consensus_queue.cc:1035] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "a71ca12d6c484662aac97beda91ec3cb" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:47:35.517624 686 consensus_queue.cc:1035] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "04d4d03f128d4489a0c106c55cb9a289" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:47:35.520396 690 mvcc.cc:205] Tried to move back new op lower bound from 7298405398594646016 to 7298405398480683008. Current Snapshot: MvccSnapshot[applied={T|T < 7298405398594646016}]
I20260619 02:47:35.520771 691 mvcc.cc:205] Tried to move back new op lower bound from 7298405398594646016 to 7298405398480683008. Current Snapshot: MvccSnapshot[applied={T|T < 7298405398594646016}]
I20260619 02:47:35.523311 692 mvcc.cc:205] Tried to move back new op lower bound from 7298405398594646016 to 7298405398480683008. Current Snapshot: MvccSnapshot[applied={T|T < 7298405398594646016}]
I20260619 02:47:35.530511 32643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:47:35.530622 32643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:47:35.531320 32643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff6273c644214a08afd3d02041441977: Sending DeleteTablet for 3 replicas of tablet 72183254e4d547b085da5c365b224252
I20260619 02:47:35.531850 307 tablet_service.cc:1514] Processing DeleteTablet for tablet 72183254e4d547b085da5c365b224252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:35 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:42564
I20260619 02:47:35.531849 441 tablet_service.cc:1514] Processing DeleteTablet for tablet 72183254e4d547b085da5c365b224252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:35 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:50746
I20260619 02:47:35.532230 575 tablet_service.cc:1514] Processing DeleteTablet for tablet 72183254e4d547b085da5c365b224252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:35 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:54030
I20260619 02:47:35.532701 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 32681
I20260619 02:47:35.538982 701 tablet_replica.cc:330] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: stopping tablet replica
I20260619 02:47:35.539110 701 raft_consensus.cc:2229] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:47:35.539245 701 raft_consensus.cc:2258] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:47:35.539718 701 ts_tablet_manager.cc:1905] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:47:35.541354 701 ts_tablet_manager.cc:1918] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:47:35.541440 701 log.cc:1196] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/72183254e4d547b085da5c365b224252
I20260619 02:47:35.541677 701 ts_tablet_manager.cc:1939] T 72183254e4d547b085da5c365b224252 P c1d9dadda26f4d6d9fb01dca2109e4ec: Deleting consensus metadata
I20260619 02:47:35.542064 32630 catalog_manager.cc:4841] TS c1d9dadda26f4d6d9fb01dca2109e4ec (127.19.113.67:37193): tablet 72183254e4d547b085da5c365b224252 (table test-table [id=0de983d3bf484be88b7d8d7639deac18]) successfully deleted
I20260619 02:47:35.542986 700 tablet_replica.cc:330] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: stopping tablet replica
I20260619 02:47:35.543111 700 raft_consensus.cc:2229] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:47:35.543226 700 raft_consensus.cc:2258] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:47:35.543664 700 ts_tablet_manager.cc:1905] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:47:35.545163 700 ts_tablet_manager.cc:1918] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:47:35.545234 700 log.cc:1196] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/72183254e4d547b085da5c365b224252
I20260619 02:47:35.545480 700 ts_tablet_manager.cc:1939] T 72183254e4d547b085da5c365b224252 P a71ca12d6c484662aac97beda91ec3cb: Deleting consensus metadata
I20260619 02:47:35.545867 32627 catalog_manager.cc:4841] TS a71ca12d6c484662aac97beda91ec3cb (127.19.113.66:40887): tablet 72183254e4d547b085da5c365b224252 (table test-table [id=0de983d3bf484be88b7d8d7639deac18]) successfully deleted
W20260619 02:47:35.547092 32630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:35483 (error 108)
I20260619 02:47:35.554997 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 377
I20260619 02:47:35.556443 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:35.556309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:35483 (local address 127.0.0.1:42572)
0619 02:47:35.556362 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:35.556407 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 02:47:35.556561 32630 catalog_manager.cc:4584] TS 04d4d03f128d4489a0c106c55cb9a289 (127.19.113.65:35483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72183254e4d547b085da5c365b224252: Network error: Client connection negotiation failed: client connection to 127.19.113.65:35483: connect: Connection refused (error 111)
I20260619 02:47:35.560971 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 510
I20260619 02:47:35.567108 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 32610
I20260619 02:47:35.573488 19909 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:47:35.907435 19909 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:47:35.907531 19909 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:47:36.241376 19909 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:47:36.241 UTC [31676] LOG: received smart shutdown request
2026-06-19 02:47:36.244 UTC [31676] LOG: background worker "logical replication launcher" (PID 31685) exited with exit code 1
2026-06-19 02:47:36.244 UTC [31680] LOG: shutting down
2026-06-19 02:47:36.388 UTC [31676] LOG: database system is shut down
2026-06-19T02:47:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46306 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:47:36 dist-test-slave-zqpb krb5kdc[711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:47:36 dist-test-slave-zqpb krb5kdc[711](info): set up 2 sockets
Jun 19 02:47:36 dist-test-slave-zqpb krb5kdc[711](info): commencing operation
krb5kdc: starting...
W20260619 02:47:37.438679 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:47:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:47:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:47:37.549157 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:38277
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:38277,127.19.113.125:46593,127.19.113.124:39447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:37.627343 724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:37.627547 724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:37.627568 724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:37.628679 724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:37.628713 724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:37.628726 724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:37.628737 724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:37.628747 724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:37.630178 724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:38277,127.19.113.125:46593,127.19.113.124:39447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:38277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:37.630383 724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:37.630558 724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:37.635097 732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:37.635100 729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:37.635288 724 server_base.cc:1034] running on GCE node
W20260619 02:47:37.635113 730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:37.635583 724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:37.635848 724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:37.637004 724 hybrid_clock.cc:648] HybridClock initialized: now 1781837257636991 us; error 29 us; skew 500 ppm
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:37.644003 724 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:47:37.644433 724 webserver.cc:460] Webserver started at http://127.19.113.126:44651/ using document root <none> and password file <none>
I20260619 02:47:37.644634 724 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:37.644688 724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:37.644812 724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:37.645834 724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "3cc38970d2d740d993a5d8c436f208eb"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "548c3df3f940d40734e5b4f5fb6dd02d"
server_key_iv: "c5047a2f087bc6eeb8fc7438e55c25bf"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.646170 724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "3cc38970d2d740d993a5d8c436f208eb"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "548c3df3f940d40734e5b4f5fb6dd02d"
server_key_iv: "c5047a2f087bc6eeb8fc7438e55c25bf"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.647468 724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:47:37.648177 739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:37.648332 724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:37.648422 724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "3cc38970d2d740d993a5d8c436f208eb"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "548c3df3f940d40734e5b4f5fb6dd02d"
server_key_iv: "c5047a2f087bc6eeb8fc7438e55c25bf"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.648478 724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:37.667778 724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:37.669914 724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:37.670047 724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:37.674054 724 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:38277
I20260619 02:47:37.674096 791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:38277 every 8 connection(s)
I20260619 02:47:37.674417 724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:47:37.675288 792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:37.676831 792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:37.683501 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 724
I20260619 02:47:37.683601 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:47:37.683797 19909 external_mini_cluster.cc:1428] Setting key 7ea617d9d36afe2d1ecf9edfd147fa07
I20260619 02:47:37.686487 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.677519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49702 (local address 127.19.113.126:38277)
0619 02:47:37.677697 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.677706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.677817 (+ 111us) server_negotiation.cc:407] Connection header received
0619 02:47:37.678353 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.678355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.678375 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.678481 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.679081 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.680038 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.680888 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.681211 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.681302 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.683252 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:37.683258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:37.683258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:37.683265 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:37.684970 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.685413 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.685415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.685415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.685497 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.685717 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.685717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.685717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.685907 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:37.686008 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:37.686233 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 02:47:37.686327 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260619 02:47:37.686553 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.677485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:38277 (local address 127.0.0.1:49702)
0619 02:47:37.677686 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.677729 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:37.678201 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:37.678486 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:37.678488 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:37.678768 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:37.678996 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.679001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.680136 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.680137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.680806 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.680808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.681282 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.681283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.681347 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.681858 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:37.681869 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:37.683177 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:37.685048 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.685057 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.685059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.685334 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.685558 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.685558 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.685559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.685664 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.686010 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:37.686016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:37.686094 (+ 78us) client_negotiation.cc:769] Sending connection context
0619 02:47:37.686201 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 02:47:37.686411 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":64,"thread_start_us":87,"threads_started":1}
I20260619 02:47:37.687281 792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:47:37.687620 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.687490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:53890)
0619 02:47:37.687541 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.687594 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 02:47:37.687752 741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
I20260619 02:47:37.688146 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.688057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:53892)
0619 02:47:37.688097 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.688125 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:47:37.688246 792 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:46593: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.19.113.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab.
Entry for principal kudu/127.19.113.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab.
I20260619 02:47:37.710021 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.125:46593
--webserver_interface=127.19.113.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab
--principal=kudu/127.19.113.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:38277,127.19.113.125:46593,127.19.113.124:39447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:47:37.737496 792 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } attempt: 1
I20260619 02:47:37.737957 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.737837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:53894)
0619 02:47:37.737896 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.737927 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:47:37.738265 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.738186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:53900)
0619 02:47:37.738222 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.738246 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 02:47:37.738358 792 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:46593: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
W20260619 02:47:37.787761 799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:37.787969 799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:37.787999 799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:37.789119 799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:37.789156 799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:37.789175 799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:37.789191 799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:37.789212 799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:37.791129 799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:38277,127.19.113.125:46593,127.19.113.124:39447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab
--principal=kudu/127.19.113.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.125:46593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.113.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:37.791359 799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:37.791746 799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:37.796388 807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:37.796388 804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:37.796425 805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:37.796646 799 server_base.cc:1034] running on GCE node
I20260619 02:47:37.796885 799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:37.797158 799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:37.798295 799 hybrid_clock.cc:648] HybridClock initialized: now 1781837257798279 us; error 29 us; skew 500 ppm
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:37.800727 799 init.cc:377] Logged in from keytab as kudu/127.19.113.125@KRBTEST.COM (short username kudu)
I20260619 02:47:37.801146 799 webserver.cc:460] Webserver started at http://127.19.113.125:36773/ using document root <none> and password file <none>
I20260619 02:47:37.801342 799 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:37.801384 799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:37.801506 799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:37.802537 799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/data/instance:
uuid: "2a2a30d71acf4b10b604cf4f7a729d0f"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "6d4ab77221730aaa62ca8bea1c8b33fc"
server_key_iv: "2e04422752cd0f2c12a9dc0427c2c6ed"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.802862 799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/wal/instance:
uuid: "2a2a30d71acf4b10b604cf4f7a729d0f"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "6d4ab77221730aaa62ca8bea1c8b33fc"
server_key_iv: "2e04422752cd0f2c12a9dc0427c2c6ed"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.804160 799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:37.804850 814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:37.805059 799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:37.805133 799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/wal
uuid: "2a2a30d71acf4b10b604cf4f7a729d0f"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "6d4ab77221730aaa62ca8bea1c8b33fc"
server_key_iv: "2e04422752cd0f2c12a9dc0427c2c6ed"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.805202 799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:37.806502 792 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } attempt: 2
I20260619 02:47:37.806955 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.806762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:43242)
0619 02:47:37.806821 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.806921 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 02:47:37.807261 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.807190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:43248)
0619 02:47:37.807222 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.807246 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 02:47:37.807343 792 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:46593: Network error: Client connection negotiation failed: client connection to 127.19.113.125:46593: connect: Connection refused (error 111)
I20260619 02:47:37.820518 799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:37.822582 799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:37.822690 799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:37.826421 799 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.125:46593
I20260619 02:47:37.826467 866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.125:46593 every 8 connection(s)
I20260619 02:47:37.826789 799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
I20260619 02:47:37.827679 867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:37.829216 867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:47:37.834734 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 799
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:37.834857 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-1/wal/instance
I20260619 02:47:37.835071 19909 external_mini_cluster.cc:1428] Setting key 47609d580b59208048e0a1c036a119d6
I20260619 02:47:37.838063 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.829898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.19.113.126:38277)
0619 02:47:37.829955 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.829957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.830127 (+ 170us) server_negotiation.cc:407] Connection header received
0619 02:47:37.830596 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.830598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.830615 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.830663 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.831335 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.831914 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.832877 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.833090 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.833144 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.835367 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:37.835368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:37.835368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:37.835376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:37.836771 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.837158 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.837159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.837160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.837217 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.837463 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.837463 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.837464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.837565 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:37.837619 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:37.837929 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 02:47:37.838000 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:47:37.838249 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.829861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:38277 (local address 127.0.0.1:34828)
0619 02:47:37.830011 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.830050 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:37.830499 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:37.830747 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:37.830751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:37.831056 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:37.831245 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.831250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.832007 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.832008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.832795 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.832797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.833159 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.833160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.833284 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.833722 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:37.833729 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:37.835275 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:37.836855 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.836862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.836864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.837089 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.837274 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.837275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.837275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.837397 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.837614 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:37.837618 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:37.837771 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 02:47:37.837873 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:47:37.838026 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260619 02:47:37.838815 867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.113.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.124@KRBTEST.COM" created.
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:47:37.848518 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.839094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:43252)
0619 02:47:37.839149 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.839162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:37.839209 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:37.842412 (+ 3203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:37.842413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:37.842520 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:37.842572 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.842573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.843430 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.843431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.843950 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.843952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.844416 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.844417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.844481 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.844913 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:37.844918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:37.846112 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:37.847640 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.847641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.847641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.847790 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.847952 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.847953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.847953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.848004 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.848230 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:37.848231 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:37.848267 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:47:37.848325 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:47:37.848420 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:47:37.848601 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.842122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.19.113.125:46593)
0619 02:47:37.842272 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.842276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.842295 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:47:37.842314 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.842316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.842326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.842388 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.842629 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.843359 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.844032 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.844335 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.844420 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.846183 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:37.846191 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:37.846191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:37.846197 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:37.847569 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.847852 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.847855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.847856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.847901 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.848049 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.848049 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.848050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.848174 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:37.848239 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:37.848368 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 02:47:37.848488 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260619 02:47:37.849160 867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:47:37.849470 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.849359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39460)
0619 02:47:37.849399 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.849444 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 02:47:37.849583 816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
I20260619 02:47:37.849990 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.849903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39466)
0619 02:47:37.849941 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.849966 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 02:47:37.850082 867 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39447: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab.
Entry for principal kudu/127.19.113.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab.
I20260619 02:47:37.859776 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.124:39447
--webserver_interface=127.19.113.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab
--principal=kudu/127.19.113.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:38277,127.19.113.125:46593,127.19.113.124:39447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:47:37.898516 792 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } attempt: 3
I20260619 02:47:37.899286 867 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } attempt: 1
I20260619 02:47:37.899642 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.899530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39476)
0619 02:47:37.899577 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.899609 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:47:37.899988 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.899861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39480)
0619 02:47:37.899910 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.899947 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 02:47:37.900116 867 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39447: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:47:37.905920 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.899100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.19.113.125:46593)
0619 02:47:37.899157 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.899159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.899170 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:47:37.899199 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.899200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.899213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.899285 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.899627 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.900322 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.901332 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.901560 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.901624 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.903639 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:37.903640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:37.903640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:37.903648 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:37.904915 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.905230 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.905231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.905232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.905284 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.905447 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.905447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.905448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.905554 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:37.905607 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:37.905758 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 02:47:37.905828 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:47:37.905946 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.898965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:43268)
0619 02:47:37.899054 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.899066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:37.899110 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:37.899300 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:37.899301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:37.899484 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:37.899558 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.899560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.900402 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.900403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.901227 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.901230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.901635 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.901636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.901714 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.902146 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:37.902151 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:37.903556 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:37.905005 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.905006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.905006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.905171 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.905337 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.905337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.905338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.905395 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.905609 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:37.905609 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:37.905659 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 02:47:37.905721 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:47:37.905820 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260619 02:47:37.906386 792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:47:37.906703 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.906608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39496)
0619 02:47:37.906646 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.906677 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 02:47:37.907014 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.906947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39502)
0619 02:47:37.906976 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.907001 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260619 02:47:37.907096 792 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39447: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
I20260619 02:47:37.938274 792 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } attempt: 1
I20260619 02:47:37.938782 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.938635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39506)
0619 02:47:37.938697 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.938745 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:47:37.939204 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.939088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39512)
0619 02:47:37.939146 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.939179 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 02:47:37.939368 792 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39447: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
W20260619 02:47:37.941278 874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:37.941534 874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:37.941588 874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:37.942790 874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:37.942828 874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:37.942847 874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:37.942862 874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:37.942898 874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:37.944409 874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:38277,127.19.113.125:46593,127.19.113.124:39447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab
--principal=kudu/127.19.113.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.124:39447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.113.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:37.944645 874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:37.944840 874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:37.949348 880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:37.949344 882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:37.949348 879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:37.949568 874 server_base.cc:1034] running on GCE node
I20260619 02:47:37.949801 874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:37.950063 874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:37.951215 874 hybrid_clock.cc:648] HybridClock initialized: now 1781837257951202 us; error 27 us; skew 500 ppm
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:37.953675 874 init.cc:377] Logged in from keytab as kudu/127.19.113.124@KRBTEST.COM (short username kudu)
I20260619 02:47:37.954110 874 webserver.cc:460] Webserver started at http://127.19.113.124:38877/ using document root <none> and password file <none>
I20260619 02:47:37.954305 874 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:37.954347 874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:37.954470 874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:37.955523 874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/data/instance:
uuid: "9ff13dae1e8140c995971e183be32826"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "d8a2472395bf65c82f90f803a260f1d1"
server_key_iv: "1f2f278ced882b2a460840904f1d018f"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.955842 874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/wal/instance:
uuid: "9ff13dae1e8140c995971e183be32826"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "d8a2472395bf65c82f90f803a260f1d1"
server_key_iv: "1f2f278ced882b2a460840904f1d018f"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.957054 874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:47:37.957813 889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:37.958005 874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:37.958089 874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/wal
uuid: "9ff13dae1e8140c995971e183be32826"
format_stamp: "Formatted at 2026-06-19 02:47:37 on dist-test-slave-zqpb"
server_key: "d8a2472395bf65c82f90f803a260f1d1"
server_key_iv: "1f2f278ced882b2a460840904f1d018f"
server_key_version: "encryptionkey@0"
I20260619 02:47:37.958160 874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:37.968266 867 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } attempt: 2
I20260619 02:47:37.968699 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.968582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39526)
0619 02:47:37.968634 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.968672 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:47:37.969029 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.968919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39542)
0619 02:47:37.968986 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.969010 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 02:47:37.969110 867 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39447: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39447: connect: Connection refused (error 111)
I20260619 02:47:37.972749 874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:37.974407 874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:37.974511 874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:37.978129 874 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.124:39447
I20260619 02:47:37.978176 941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.124:39447 every 8 connection(s)
I20260619 02:47:37.978487 874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
I20260619 02:47:37.979390 942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:37.980898 942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:47:37.984498 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 874
I20260619 02:47:37.984638 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837111403550-19909-0/minicluster-data/master-2/wal/instance
I20260619 02:47:37.984849 19909 external_mini_cluster.cc:1428] Setting key f2886d09bf954fe205bad229884adbfb
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:47:37.990558 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.981557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34830 (local address 127.19.113.126:38277)
0619 02:47:37.981615 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.981617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.981781 (+ 164us) server_negotiation.cc:407] Connection header received
0619 02:47:37.982332 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.982333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.982344 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.982421 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.983115 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.983601 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.984935 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.985142 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.985216 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.987583 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:37.987584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:37.987584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:37.987592 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:37.989184 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.989583 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.989585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.989585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.989647 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.989926 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.989926 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.989926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.990030 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:37.990076 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:37.990377 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 02:47:37.990466 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:47:37.990628 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.981508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:38277 (local address 127.0.0.1:34830)
0619 02:47:37.981672 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.981713 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:37.982183 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:37.982530 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:37.982534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:37.982778 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:37.983020 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.983025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.983681 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.983682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.984841 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.984843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.985231 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.985232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.985393 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.985853 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:37.985863 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:37.987500 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:37.989270 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.989276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.989278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.989509 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.989717 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.989718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.989718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.989848 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.990084 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:37.990090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:37.990234 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:47:37.990323 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:47:37.990448 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260619 02:47:37.991128 942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:47:37.993708 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.985277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39552 (local address 127.19.113.124:39447)
0619 02:47:37.985435 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.985439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.985454 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:47:37.985472 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.985473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.985489 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.985554 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.986224 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.987199 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.987894 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.988223 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.988278 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.990602 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:37.990608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:37.990609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:37.990615 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:37.992348 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.992704 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.992707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.992707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.992763 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.992975 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.992976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.992977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.993209 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:37.993408 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:37.993523 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 02:47:37.993609 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:47:37.997843 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.991462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:46593 (local address 127.0.0.1:43284)
0619 02:47:37.991511 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.991521 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:37.991558 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:37.991671 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:37.991672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:37.991805 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:37.991836 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.991838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.992516 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.992517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.993032 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.993035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.993498 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.993499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.993557 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.993917 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:37.993920 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:37.995343 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:37.996856 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.996856 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.996857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.997044 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.997232 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:37.997233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:37.997233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:37.997295 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:37.997544 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:37.997545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:37.997587 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:47:37.997641 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:47:37.997748 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:47:37.997864 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.991526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.19.113.125:46593)
0619 02:47:37.991570 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.991572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.991581 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:47:37.991616 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.991617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.991624 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.991668 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.991923 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.992440 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.993138 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.993429 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.993500 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.995435 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:37.995436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:37.995436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:37.995444 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:37.996776 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.997106 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.997107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.997108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.997167 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:37.997358 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:37.997360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:37.997360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:37.997484 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:37.997546 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:37.997702 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:47:37.997773 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:47:37.998263 942 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:47:38.001700 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.994637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.19.113.126:38277)
0619 02:47:37.994695 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.994697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.994730 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:47:37.994790 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.994791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.994801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.994850 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.995424 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.996071 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.996826 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.997047 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.997102 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:37.999192 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:37.999193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:37.999194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:37.999201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:38.000552 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.000887 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.000888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.000888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.000946 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.001124 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.001125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.001125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.001259 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:38.001487 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:38.001533 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 02:47:38.001611 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:47:38.004369 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.998521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39556)
0619 02:47:37.998551 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:37.998560 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:37.998597 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:37.998714 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:37.998715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:37.998835 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:37.998886 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.998888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.999488 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:37.999489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:37.999939 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:37.999940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.000257 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.000258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:38.000300 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.000619 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:38.000624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:38.001928 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:38.003402 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:38.003403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:38.003403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:38.003597 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:38.003820 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:38.003820 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:38.003821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:38.003868 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:38.004128 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:38.004129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:38.004166 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:47:38.004224 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:47:38.004297 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 02:47:38.004475 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:37.998547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.19.113.124:39447)
0619 02:47:37.998646 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:37.998647 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:37.998651 (+ 4us) server_negotiation.cc:407] Connection header received
0619 02:47:37.998662 (+ 11us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:37.998662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:37.998669 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:37.998703 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:37.998980 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:37.999425 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.000000 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.000205 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.000254 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.001989 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:38.001990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:38.001991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:38.001997 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:38.003324 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.003687 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.003688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.003688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.003750 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.003916 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.003917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.003918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.004049 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:38.004112 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:38.004258 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 02:47:38.004368 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":90}
I20260619 02:47:38.006043 942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826: Bootstrap starting.
Jun 19 02:47:37 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:47:38.006990 942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:38.007315 942 log.cc:824] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:38.008070 942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826: No bootstrap required, opened a new log
I20260619 02:47:38.009974 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:38.002359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.19.113.125:46593)
0619 02:47:38.002405 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:38.002407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:38.002414 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:47:38.002455 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:38.002456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:38.002466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:38.002504 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:38.003040 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.003601 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.004585 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.004940 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.004994 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.007246 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:38.007247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:38.007247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:38.007253 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:38.009020 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.009365 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.009366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.009366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.009412 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.009583 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.009583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.009584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.009670 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:38.009779 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:38.009854 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:47:38.009914 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:47:38.009924 942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.010066 942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:38.010102 942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff13dae1e8140c995971e183be32826, State: Initialized, Role: FOLLOWER
I20260619 02:47:38.010202 942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.010456 948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.010502 942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:38.010551 948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:47:38.011564 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 02:47:38.012231 959 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:47:38.012346 959 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:47:38.015069 792 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } attempt: 2
I20260619 02:47:38.013542 874 master_runner.cc:186] Error getting master registration for 127.19.113.126:38277: OK, instance_id { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" instance_seqno: 1781837257672889 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:47:38.015802 874 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 19 02:47:38 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:47:38.021272 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:38.013124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.19.113.126:38277)
0619 02:47:38.013159 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:38.013160 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:38.013332 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:47:38.013419 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:38.013420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:38.013430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:38.013457 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:38.014171 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.015018 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.016383 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.016650 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.016704 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.018140 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:38.018141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:38.018142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:38.018149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:38.020007 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.020368 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.020369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.020370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.020422 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.020601 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.020602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.020602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.020720 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:38.020814 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:38.021104 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 02:47:38.021178 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:47:38.022037 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:38.013070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.19.113.125:46593)
0619 02:47:38.013107 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:38.013108 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:38.013632 (+ 524us) server_negotiation.cc:407] Connection header received
0619 02:47:38.013717 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:38.013717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:38.013727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:38.013753 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:38.014294 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.014998 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.016082 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.016377 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.016442 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.018128 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:38.018129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:38.018129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:38.018135 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:38.020537 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.020990 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.020991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.020991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.021048 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.021239 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.021240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.021240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.021361 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:38.021648 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:38.021857 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:47:38.021949 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:47:38.024698 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:38.015295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39570)
0619 02:47:38.015343 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:38.015355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:38.015399 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:38.016518 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:38.016519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:38.016680 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:38.016719 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:38.016722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.017684 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.017685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:38.018394 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:38.018397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.018917 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.018919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:38.019029 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.019391 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:38.019395 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:38.020948 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:38.023770 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:38.023771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:38.023771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:38.023967 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:38.024135 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:38.024136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:38.024136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:38.024195 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:38.024444 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:38.024445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:38.024485 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 02:47:38.024546 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:47:38.024633 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:47:38.024786 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:38.015515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.19.113.124:39447)
0619 02:47:38.016414 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:38.016415 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:38.016421 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:47:38.016439 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:38.016439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:38.016448 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:38.016508 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:38.016801 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.017584 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.018506 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.018805 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.019165 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.021025 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:38.021026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:38.021027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:38.021033 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:38.023677 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.024026 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.024026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.024027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.024078 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.024258 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.024259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.024259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.024390 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:38.024452 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:38.024591 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 02:47:38.024669 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":68,"threads_started":1}
I20260619 02:47:38.025211 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:38.013360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39558 (local address 127.19.113.124:39447)
0619 02:47:38.013393 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:38.013395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:38.016633 (+ 3238us) server_negotiation.cc:407] Connection header received
0619 02:47:38.016716 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:38.016717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:38.016726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:38.016752 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:38.017465 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.018127 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.019026 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.019216 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.019268 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.022309 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:38.022310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:38.022310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:38.022317 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:38.023819 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.024172 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.024173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.024173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.024211 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.024386 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.024387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.024387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.024498 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:38.024583 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:38.025060 (+ 477us) server_negotiation.cc:299] Negotiation successful
0619 02:47:38.025132 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:47:38.026116 792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb: Bootstrap starting.
I20260619 02:47:38.026721 792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:38.027019 792 log.cc:824] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:38.027709 792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb: No bootstrap required, opened a new log
I20260619 02:47:38.029331 792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.029443 792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:38.029487 792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cc38970d2d740d993a5d8c436f208eb, State: Initialized, Role: FOLLOWER
I20260619 02:47:38.029605 792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.029852 970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.029920 792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:38.029934 970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:47:38.031826 981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:47:38.031927 981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:47:38.032301 724 master_runner.cc:186] Error getting master registration for 127.19.113.125:46593: OK, instance_id { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" instance_seqno: 1781837257825408 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:47:38.035079 724 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:47:38.060245 867 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } attempt: 3
Jun 19 02:47:38 dist-test-slave-zqpb krb5kdc[711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837257, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:47:38.068578 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:38.060523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39447 (local address 127.0.0.1:39576)
0619 02:47:38.060572 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:38.060582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:38.060618 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:38.060756 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:38.060757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:38.060941 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:38.060996 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:38.060998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.061926 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.061928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:38.062741 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:38.062743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.063223 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.063224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:38.063285 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.063798 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:38.063803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:38.065345 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:38.067132 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:38.067133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:38.067133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:38.067349 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:38.067581 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:38.067583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:38.067583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:38.067646 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:38.068229 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:38.068230 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:38.068284 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 02:47:38.068362 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:47:38.068475 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:47:38.068575 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:38.060603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.19.113.124:39447)
0619 02:47:38.060647 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:38.060649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:38.060657 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:47:38.060678 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:38.060678 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:38.060688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:38.060744 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:38.061066 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.061843 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.062829 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:38.063149 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:38.063216 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:38.065437 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:38.065438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:38.065439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:38.065446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:38.067041 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.067423 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.067424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.067425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.067503 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:38.068004 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:38.068005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:38.068005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:38.068158 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:38.068228 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:38.068394 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 02:47:38.068475 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:47:38.070431 867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f: Bootstrap starting.
I20260619 02:47:38.071190 867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:38.071501 867 log.cc:824] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:38.071847 970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:38.071944 970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.072180 970 leader_election.cc:288] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a2a30d71acf4b10b604cf4f7a729d0f (127.19.113.125:46593), 9ff13dae1e8140c995971e183be32826 (127.19.113.124:39447)
I20260619 02:47:38.072351 867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f: No bootstrap required, opened a new log
I20260619 02:47:38.072781 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cc38970d2d740d993a5d8c436f208eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ff13dae1e8140c995971e183be32826" is_pre_election: true
I20260619 02:47:38.072908 916 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cc38970d2d740d993a5d8c436f208eb in term 0.
I20260619 02:47:38.073124 741 leader_election.cc:302] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cc38970d2d740d993a5d8c436f208eb, 9ff13dae1e8140c995971e183be32826; no voters:
I20260619 02:47:38.073246 970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:47:38.073295 970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:47:38.073321 970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:38.074133 970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.074254 970 leader_election.cc:288] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [CANDIDATE]: Term 1 election: Requested vote from peers 2a2a30d71acf4b10b604cf4f7a729d0f (127.19.113.125:46593), 9ff13dae1e8140c995971e183be32826 (127.19.113.124:39447)
I20260619 02:47:38.074653 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cc38970d2d740d993a5d8c436f208eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ff13dae1e8140c995971e183be32826"
I20260619 02:47:38.074731 916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:38.074698 867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.074848 867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:38.074911 867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2a30d71acf4b10b604cf4f7a729d0f, State: Initialized, Role: FOLLOWER
I20260619 02:47:38.075047 867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.075364 867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:38.075465 916 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cc38970d2d740d993a5d8c436f208eb in term 1.
I20260619 02:47:38.075570 840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cc38970d2d740d993a5d8c436f208eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f"
I20260619 02:47:38.075659 840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:38.076412 840 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cc38970d2d740d993a5d8c436f208eb in term 1.
I20260619 02:47:38.076843 741 leader_election.cc:302] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cc38970d2d740d993a5d8c436f208eb, 9ff13dae1e8140c995971e183be32826; no voters:
I20260619 02:47:38.076670 841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cc38970d2d740d993a5d8c436f208eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" is_pre_election: true
I20260619 02:47:38.077049 841 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3cc38970d2d740d993a5d8c436f208eb in term 1. Re-sending same reply.
I20260619 02:47:38.077128 970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:47:38.077276 970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [term 1 LEADER]: Becoming Leader. State: Replica: 3cc38970d2d740d993a5d8c436f208eb, State: Running, Role: LEADER
I20260619 02:47:38.077361 970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } }
I20260619 02:47:38.077749 970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc38970d2d740d993a5d8c436f208eb. Latest consensus state: current_term: 1 leader_uuid: "3cc38970d2d740d993a5d8c436f208eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.077816 970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:38.077992 998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:47:38.078219 998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260619 02:47:38.080087 995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:47:38.080205 995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:47:38.081321 841 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [term 1 FOLLOWER]: Refusing update from remote peer 3cc38970d2d740d993a5d8c436f208eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:47:38.081326 916 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [term 1 FOLLOWER]: Refusing update from remote peer 3cc38970d2d740d993a5d8c436f208eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:47:38.081631 970 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:47:38.081785 970 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:47:38.083657 948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc38970d2d740d993a5d8c436f208eb. Latest consensus state: current_term: 1 leader_uuid: "3cc38970d2d740d993a5d8c436f208eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.083748 948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:47:38.084352 1001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cc38970d2d740d993a5d8c436f208eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.084435 1001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:47:38.084635 984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc38970d2d740d993a5d8c436f208eb. Latest consensus state: current_term: 1 leader_uuid: "3cc38970d2d740d993a5d8c436f208eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.084741 984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:47:38.084982 970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3cc38970d2d740d993a5d8c436f208eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.085062 970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:38.085125 1000 mvcc.cc:205] Tried to move back new op lower bound from 7298405409098956800 to 7298405409085411328. Current Snapshot: MvccSnapshot[applied={T|T < 7298405409098956800}]
I20260619 02:47:38.085203 948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3cc38970d2d740d993a5d8c436f208eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.085278 948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:47:38.085604 1002 mvcc.cc:205] Tried to move back new op lower bound from 7298405409098956800 to 7298405409085411328. Current Snapshot: MvccSnapshot[applied={T|T < 7298405409098956800}]
I20260619 02:47:38.085794 998 catalog_manager.cc:1347] Generated new cluster ID: 9ae1d6bc1d344857b21c87da4a72079c
I20260619 02:47:38.085850 998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:47:38.092078 996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3cc38970d2d740d993a5d8c436f208eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.092170 996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:38.092860 1004 mvcc.cc:205] Tried to move back new op lower bound from 7298405409098956800 to 7298405409085411328. Current Snapshot: MvccSnapshot[applied={T|T < 7298405409098956800}]
I20260619 02:47:38.095324 984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3cc38970d2d740d993a5d8c436f208eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cc38970d2d740d993a5d8c436f208eb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38277 } } peers { permanent_uuid: "2a2a30d71acf4b10b604cf4f7a729d0f" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 46593 } } peers { permanent_uuid: "9ff13dae1e8140c995971e183be32826" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39447 } } }
I20260619 02:47:38.095477 984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:47:38.112262 998 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:47:38.113080 998 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:47:38.120378 998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cc38970d2d740d993a5d8c436f208eb: Generated new TSK 0
I20260619 02:47:38.120641 998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:47:39.012832 959 catalog_manager.cc:1259] Loaded cluster ID: 9ae1d6bc1d344857b21c87da4a72079c
I20260619 02:47:39.012903 959 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826: loading cluster ID for follower catalog manager: success
I20260619 02:47:39.013777 959 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826: acquiring CA information for follower catalog manager: success
I20260619 02:47:39.014055 959 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9ff13dae1e8140c995971e183be32826: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:47:39.080680 995 catalog_manager.cc:1259] Loaded cluster ID: 9ae1d6bc1d344857b21c87da4a72079c
I20260619 02:47:39.080757 995 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f: loading cluster ID for follower catalog manager: success
I20260619 02:47:39.081526 995 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f: acquiring CA information for follower catalog manager: success
I20260619 02:47:39.081779 995 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a2a30d71acf4b10b604cf4f7a729d0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:47:39.177464 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 724
I20260619 02:47:39.181986 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 799
I20260619 02:47:39.186271 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 874
2026-06-19T02:47:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2711 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:47:39 dist-test-slave-zqpb krb5kdc[1015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:47:39 dist-test-slave-zqpb krb5kdc[1015](info): set up 2 sockets
Jun 19 02:47:39 dist-test-slave-zqpb krb5kdc[1015](info): commencing operation
krb5kdc: starting...
W20260619 02:47:40.142755 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:47:40 dist-test-slave-zqpb krb5kdc[1015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:47:40.251356 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:34181
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:34181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:40.331933 1028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:40.332145 1028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:40.332170 1028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:40.333323 1028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:40.333365 1028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:40.333379 1028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:40.333389 1028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:40.333400 1028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:40.334960 1028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:34181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:34181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:40.335183 1028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:40.335405 1028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:40.339963 1033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:40.339948 1036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:40.340057 1028 server_base.cc:1034] running on GCE node
W20260619 02:47:40.339941 1034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:40.340390 1028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:40.340652 1028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:40.341789 1028 hybrid_clock.cc:648] HybridClock initialized: now 1781837260341776 us; error 28 us; skew 500 ppm
Jun 19 02:47:40 dist-test-slave-zqpb krb5kdc[1015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837260, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:40.344131 1028 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:47:40.344489 1028 webserver.cc:460] Webserver started at http://127.19.113.126:34239/ using document root <none> and password file <none>
I20260619 02:47:40.344663 1028 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:40.344702 1028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:40.344789 1028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:40.345781 1028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "45440266d8d74a8291f7748b22c432fc"
format_stamp: "Formatted at 2026-06-19 02:47:40 on dist-test-slave-zqpb"
server_key: "9ca3d57d9c3b80563fea346ca23ff6ee"
server_key_iv: "edd713d3245b52ac3f3dd8704d42672a"
server_key_version: "encryptionkey@0"
I20260619 02:47:40.346077 1028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "45440266d8d74a8291f7748b22c432fc"
format_stamp: "Formatted at 2026-06-19 02:47:40 on dist-test-slave-zqpb"
server_key: "9ca3d57d9c3b80563fea346ca23ff6ee"
server_key_iv: "edd713d3245b52ac3f3dd8704d42672a"
server_key_version: "encryptionkey@0"
I20260619 02:47:40.347265 1028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:47:40.347908 1043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:40.348069 1028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:40.348151 1028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "45440266d8d74a8291f7748b22c432fc"
format_stamp: "Formatted at 2026-06-19 02:47:40 on dist-test-slave-zqpb"
server_key: "9ca3d57d9c3b80563fea346ca23ff6ee"
server_key_iv: "edd713d3245b52ac3f3dd8704d42672a"
server_key_version: "encryptionkey@0"
I20260619 02:47:40.348209 1028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:40.369133 1028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:40.370850 1028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:40.371002 1028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:40.374513 1028 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:34181
I20260619 02:47:40.374562 1095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:34181 every 8 connection(s)
I20260619 02:47:40.374887 1028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:47:40.375731 1096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:40.375779 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 1028
I20260619 02:47:40.375859 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:47:40.376044 19909 external_mini_cluster.cc:1428] Setting key b689ff57b611aa7c15c01e468815dcc4
I20260619 02:47:40.378899 1096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc: Bootstrap starting.
I20260619 02:47:40.379544 1096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:40.379801 1096 log.cc:824] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:40.380461 1096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc: No bootstrap required, opened a new log
Jun 19 02:47:40 dist-test-slave-zqpb krb5kdc[1015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:40.382552 1096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45440266d8d74a8291f7748b22c432fc" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34181 } }
I20260619 02:47:40.382709 1096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:40.382752 1096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45440266d8d74a8291f7748b22c432fc, State: Initialized, Role: FOLLOWER
I20260619 02:47:40.382898 1096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45440266d8d74a8291f7748b22c432fc" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34181 } }
I20260619 02:47:40.382977 1096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:47:40.383006 1096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:47:40.383052 1096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:40.383701 1096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45440266d8d74a8291f7748b22c432fc" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34181 } }
I20260619 02:47:40.383813 1096 leader_election.cc:302] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45440266d8d74a8291f7748b22c432fc; no voters:
I20260619 02:47:40.383980 1096 leader_election.cc:288] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:47:40.384028 1101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:47:40.384186 1101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [term 1 LEADER]: Becoming Leader. State: Replica: 45440266d8d74a8291f7748b22c432fc, State: Running, Role: LEADER
I20260619 02:47:40.384230 1096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:40.384370 1101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45440266d8d74a8291f7748b22c432fc" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34181 } }
I20260619 02:47:40.384752 1102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45440266d8d74a8291f7748b22c432fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45440266d8d74a8291f7748b22c432fc" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34181 } } }
I20260619 02:47:40.384773 1103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45440266d8d74a8291f7748b22c432fc. Latest consensus state: current_term: 1 leader_uuid: "45440266d8d74a8291f7748b22c432fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45440266d8d74a8291f7748b22c432fc" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34181 } } }
I20260619 02:47:40.384898 1102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:40.384907 1103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:40.385118 1110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:47:40.385660 1110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:47:40.386755 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:40.376501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.19.113.126:34181)
0619 02:47:40.376684 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:40.376694 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:40.376726 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:47:40.377181 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:40.377191 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:40.377253 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:40.377398 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:40.377949 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:40.379290 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:40.380111 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:40.380566 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:40.380668 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:40.383215 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:40.383219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:40.383219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:40.383231 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:40.385023 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:40.385411 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:40.385414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:40.385414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:40.385518 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:40.385718 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:40.385718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:40.385719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:40.385993 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:40.386134 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:40.386447 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 02:47:40.386582 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260619 02:47:40.387310 1110 catalog_manager.cc:1347] Generated new cluster ID: feba6079ecb2413999f45d91d7e52ead
I20260619 02:47:40.387358 1110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:47:40.387759 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:47:40.402663 1110 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:47:40.403416 1110 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:47:40.409940 1110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45440266d8d74a8291f7748b22c432fc: Generated new TSK 0
I20260619 02:47:40.410125 1110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:47:40.441277 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 1028
2026-06-19T02:47:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1253 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:47:40 dist-test-slave-zqpb krb5kdc[1123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:47:40 dist-test-slave-zqpb krb5kdc[1123](info): set up 2 sockets
Jun 19 02:47:40 dist-test-slave-zqpb krb5kdc[1123](info): commencing operation
krb5kdc: starting...
W20260619 02:47:41.386701 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:47:41 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:47:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:47:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:47:41.495560 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46823
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:41.572144 1136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:41.572347 1136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:41.572377 1136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:41.573488 1136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:41.573530 1136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:41.573549 1136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:41.573565 1136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:41.573580 1136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:41.575039 1136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:46823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:41.575273 1136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:41.575465 1136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:41.579814 1144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:41.579962 1136 server_base.cc:1034] running on GCE node
W20260619 02:47:41.579823 1141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:41.579826 1142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:41.580293 1136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:41.580554 1136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:41.581703 1136 hybrid_clock.cc:648] HybridClock initialized: now 1781837261581688 us; error 27 us; skew 500 ppm
Jun 19 02:47:41 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:41.584029 1136 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:47:41.584434 1136 webserver.cc:460] Webserver started at http://127.19.113.126:37273/ using document root <none> and password file <none>
I20260619 02:47:41.584621 1136 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:41.584661 1136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:41.584750 1136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:41.585733 1136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "0d128cf3bf674b5fafb7a6897e745a4a"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "e04016f60f034d1510d0795994636bd7"
server_key_iv: "65b023bcc802be647f3e8b744f299798"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.586023 1136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "0d128cf3bf674b5fafb7a6897e745a4a"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "e04016f60f034d1510d0795994636bd7"
server_key_iv: "65b023bcc802be647f3e8b744f299798"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.587230 1136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:47:41.587877 1151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:41.588029 1136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:47:41.588105 1136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "0d128cf3bf674b5fafb7a6897e745a4a"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "e04016f60f034d1510d0795994636bd7"
server_key_iv: "65b023bcc802be647f3e8b744f299798"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.588160 1136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:41.598666 1136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:41.600651 1136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:41.600754 1136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:41.604326 1136 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:46823
I20260619 02:47:41.604357 1203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:46823 every 8 connection(s)
I20260619 02:47:41.604650 1136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:47:41.605559 1204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:41.607796 1204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Bootstrap starting.
I20260619 02:47:41.608366 1204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:41.608623 1204 log.cc:824] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:41.609266 1204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: No bootstrap required, opened a new log
I20260619 02:47:41.609504 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 1136
I20260619 02:47:41.609584 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:47:41.609740 19909 external_mini_cluster.cc:1428] Setting key ca6a3cdc2529673f3afa5373be4941fd
I20260619 02:47:41.611331 1204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:41.611503 1204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:41.611537 1204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d128cf3bf674b5fafb7a6897e745a4a, State: Initialized, Role: FOLLOWER
I20260619 02:47:41.611682 1204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:41.611757 1204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:47:41.611779 1204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:47:41.611832 1204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:41.612819 1204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:41.612933 1204 leader_election.cc:302] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d128cf3bf674b5fafb7a6897e745a4a; no voters:
I20260619 02:47:41.613072 1204 leader_election.cc:288] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:47:41.613112 1209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:47:41.613247 1209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 1 LEADER]: Becoming Leader. State: Replica: 0d128cf3bf674b5fafb7a6897e745a4a, State: Running, Role: LEADER
I20260619 02:47:41.613297 1204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:41.613355 1209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:41.613768 1211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d128cf3bf674b5fafb7a6897e745a4a. Latest consensus state: current_term: 1 leader_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } } }
I20260619 02:47:41.613842 1211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:41.614068 1210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } } }
I20260619 02:47:41.614332 1218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:47:41.614428 1210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:41.614981 1218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:47:41.616866 1218 catalog_manager.cc:1347] Generated new cluster ID: 23ead91f36ce4d6c886cdde9746b4c17
I20260619 02:47:41.616912 1218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:47:41 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:41.621730 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:41.610194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.19.113.126:46823)
0619 02:47:41.610391 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:41.610401 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:41.610437 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:47:41.611295 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:41.611305 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:41.611377 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:41.611527 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:41.612182 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.613506 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.614519 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.615078 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.615176 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:41.617707 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:41.617711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:41.617711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:41.617723 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:41.619705 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:41.620090 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:41.620093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:41.620093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:41.620215 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:41.620424 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:41.620425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:41.620425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:41.620717 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:41.620857 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:41.621381 (+ 524us) server_negotiation.cc:299] Negotiation successful
0619 02:47:41.621532 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260619 02:47:41.626945 1218 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:47:41.627581 1218 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:47:41.636854 1218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Generated new TSK 0
I20260619 02:47:41.637080 1218 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:47:41.666764 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46823
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:41.744864 1230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:41.745070 1230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:41.745092 1230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:41.746191 1230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:41.746225 1230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:41.746248 1230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:47:41.747696 1230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:41.747884 1230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:41.748061 1230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:47:41.752655 1230 server_base.cc:1034] running on GCE node
W20260619 02:47:41.752643 1235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:41.752640 1238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:41.752630 1236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:41.753067 1230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:41.753279 1230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:41.754422 1230 hybrid_clock.cc:648] HybridClock initialized: now 1781837261754387 us; error 50 us; skew 500 ppm
Jun 19 02:47:41 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:41.756747 1230 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:47:41.757182 1230 webserver.cc:460] Webserver started at http://127.19.113.65:44685/ using document root <none> and password file <none>
I20260619 02:47:41.757387 1230 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:41.757427 1230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:41.757557 1230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:41.758603 1230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "97102225dc5f49b599d442616e438c2e"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "7ee1631d2f82a77cfe56fd64f66f8c1b"
server_key_iv: "d7888b74df2fc31411281941d2911977"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.758960 1230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "97102225dc5f49b599d442616e438c2e"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "7ee1631d2f82a77cfe56fd64f66f8c1b"
server_key_iv: "d7888b74df2fc31411281941d2911977"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.760169 1230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:41.760788 1245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:41.760917 1230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:41.760999 1230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "97102225dc5f49b599d442616e438c2e"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "7ee1631d2f82a77cfe56fd64f66f8c1b"
server_key_iv: "d7888b74df2fc31411281941d2911977"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.761097 1230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:41.779645 1230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:41.781410 1230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:41.781556 1230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:41.781759 1230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:41.782099 1230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:41.782140 1230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:41.782169 1230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:41.782197 1230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:41.787516 1230 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:39997
I20260619 02:47:41.787560 1358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:39997 every 8 connection(s)
I20260619 02:47:41.787847 1230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:47:41.791513 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 1230
I20260619 02:47:41.791595 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:47:41.791772 19909 external_mini_cluster.cc:1428] Setting key 54cb493705a88d56d47cd74edc45a631
Jun 19 02:47:41 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:41.797101 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:41.788507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:53321 (local address 127.19.113.126:46823)
0619 02:47:41.788562 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:41.788564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:41.788731 (+ 167us) server_negotiation.cc:407] Connection header received
0619 02:47:41.789218 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:41.789219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:41.789233 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:41.789290 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:41.790246 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.790771 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.791969 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.792240 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.792316 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:41.794443 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:41.794444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:41.794445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:41.794452 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:41.795798 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:41.796203 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:41.796205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:41.796205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:41.796269 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:41.796478 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:41.796478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:41.796479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:41.796603 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:41.796656 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:41.796965 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 02:47:41.797036 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:47:41.797386 1361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:41.788451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:53321)
0619 02:47:41.788612 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:41.788652 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:41.789135 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:41.789352 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:41.789355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:41.789929 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:41.790169 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:41.790175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.790837 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.790838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:41.791876 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:41.791879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.792332 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.792332 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:41.792468 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:41.792910 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:41.792917 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:41.794335 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:41.795891 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:41.795898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:41.795901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:41.796134 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:41.796329 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:41.796330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:41.796330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:41.796420 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:41.796663 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:41.796670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:41.796836 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 02:47:41.796954 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 02:47:41.797150 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260619 02:47:41.798028 1359 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46823
I20260619 02:47:41.798120 1359 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:41.798285 1359 heartbeater.cc:511] Master 127.19.113.126:46823 requested a full tablet report, sending...
I20260619 02:47:41.798794 1168 ts_manager.cc:194] Registered new tserver with Master: 97102225dc5f49b599d442616e438c2e (127.19.113.65:39997)
I20260619 02:47:41.799392 1168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:53321
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:47:41.815683 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46823
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:41.894002 1364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:41.894194 1364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:41.894217 1364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:41.895331 1364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:41.895366 1364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:41.895390 1364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:47:41.896772 1364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:41.896950 1364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:41.897136 1364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:41.901778 1372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:41.901793 1370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:41.901809 1369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:41.901865 1364 server_base.cc:1034] running on GCE node
I20260619 02:47:41.902247 1364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:41.902456 1364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:41.903584 1364 hybrid_clock.cc:648] HybridClock initialized: now 1781837261903567 us; error 32 us; skew 500 ppm
Jun 19 02:47:41 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:41.905830 1364 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:47:41.906255 1364 webserver.cc:460] Webserver started at http://127.19.113.66:36687/ using document root <none> and password file <none>
I20260619 02:47:41.906442 1364 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:41.906484 1364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:41.906598 1364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:41.907639 1364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "635262b47d2940198774651e5d93abcf"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "678ac7be66d0c4187e0e8e256ea55746"
server_key_iv: "88013af157c6f53434e239159b27fd19"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.907967 1364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "635262b47d2940198774651e5d93abcf"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "678ac7be66d0c4187e0e8e256ea55746"
server_key_iv: "88013af157c6f53434e239159b27fd19"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.909157 1364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:47:41.909785 1379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:41.909940 1364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:47:41.910041 1364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "635262b47d2940198774651e5d93abcf"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "678ac7be66d0c4187e0e8e256ea55746"
server_key_iv: "88013af157c6f53434e239159b27fd19"
server_key_version: "encryptionkey@0"
I20260619 02:47:41.910094 1364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:41.917321 1364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:41.919232 1364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:41.919339 1364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:41.919503 1364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:41.919764 1364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:41.919804 1364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:41.919826 1364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:41.919874 1364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:41.925246 1364 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:42321
I20260619 02:47:41.925279 1492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:42321 every 8 connection(s)
I20260619 02:47:41.925562 1364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:47:41.929878 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 1364
I20260619 02:47:41.929975 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:47:41.930132 19909 external_mini_cluster.cc:1428] Setting key 4da0ed944cfaee325424a40f448f7d6c
Jun 19 02:47:41 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:41.934538 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:41.926186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:55977 (local address 127.19.113.126:46823)
0619 02:47:41.926242 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:41.926244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:41.926438 (+ 194us) server_negotiation.cc:407] Connection header received
0619 02:47:41.926958 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:41.926960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:41.926973 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:41.927033 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:41.927913 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.928400 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.929367 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.929565 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.929623 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:41.932030 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:41.932032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:41.932033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:41.932040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:41.933276 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:41.933662 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:41.933663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:41.933663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:41.933718 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:41.933932 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:41.933933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:41.933933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:41.934047 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:41.934101 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:41.934364 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 02:47:41.934451 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:47:41.934705 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:41.926161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:55977)
0619 02:47:41.926307 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:41.926345 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:41.926827 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:41.927099 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:41.927103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:41.927641 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:41.927820 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:41.927824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.928481 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.928482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:41.929279 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:41.929281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:41.929633 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:41.929634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:41.929776 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:41.930300 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:41.930308 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:41.931937 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:41.933377 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:41.933384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:41.933386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:41.933597 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:41.933768 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:41.933768 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:41.933769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:41.933878 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:41.934098 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:41.934105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:41.934244 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:47:41.934332 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:47:41.934495 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260619 02:47:41.935181 1493 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46823
I20260619 02:47:41.935285 1493 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:41.935473 1493 heartbeater.cc:511] Master 127.19.113.126:46823 requested a full tablet report, sending...
I20260619 02:47:41.935887 1168 ts_manager.cc:194] Registered new tserver with Master: 635262b47d2940198774651e5d93abcf (127.19.113.66:42321)
I20260619 02:47:41.936249 1168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:55977
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:47:41.951711 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46823
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:42.031405 1498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:42.031611 1498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:42.031634 1498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:42.032744 1498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:42.032783 1498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:42.032816 1498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:47:42.034214 1498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:42.034404 1498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:42.034590 1498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:42.039469 1503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.039538 1506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.039588 1504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:42.039757 1498 server_base.cc:1034] running on GCE node
I20260619 02:47:42.039937 1498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:42.040161 1498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:42.041297 1498 hybrid_clock.cc:648] HybridClock initialized: now 1781837262041284 us; error 30 us; skew 500 ppm
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:42.043678 1498 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:47:42.044082 1498 webserver.cc:460] Webserver started at http://127.19.113.67:33777/ using document root <none> and password file <none>
I20260619 02:47:42.044288 1498 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:42.044334 1498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:42.044454 1498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:42.045480 1498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "344aa666d8374552810e4a83c80d5e38"
format_stamp: "Formatted at 2026-06-19 02:47:42 on dist-test-slave-zqpb"
server_key: "42567174a32fc86644dea421a039cac8"
server_key_iv: "442e8ae62ec257db258feaf2f9eb1df5"
server_key_version: "encryptionkey@0"
I20260619 02:47:42.045799 1498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "344aa666d8374552810e4a83c80d5e38"
format_stamp: "Formatted at 2026-06-19 02:47:42 on dist-test-slave-zqpb"
server_key: "42567174a32fc86644dea421a039cac8"
server_key_iv: "442e8ae62ec257db258feaf2f9eb1df5"
server_key_version: "encryptionkey@0"
I20260619 02:47:42.046983 1498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:47:42.047617 1513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.047757 1498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:42.047843 1498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "344aa666d8374552810e4a83c80d5e38"
format_stamp: "Formatted at 2026-06-19 02:47:42 on dist-test-slave-zqpb"
server_key: "42567174a32fc86644dea421a039cac8"
server_key_iv: "442e8ae62ec257db258feaf2f9eb1df5"
server_key_version: "encryptionkey@0"
I20260619 02:47:42.047907 1498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:42.064261 1498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:42.066335 1498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:42.066468 1498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:42.066697 1498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:42.067055 1498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:42.067092 1498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.067113 1498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:42.067123 1498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.072398 1498 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:33609
I20260619 02:47:42.072436 1626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:33609 every 8 connection(s)
I20260619 02:47:42.072731 1498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:47:42.075992 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 1498
I20260619 02:47:42.076072 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:47:42.076222 19909 external_mini_cluster.cc:1428] Setting key 687c5b5e8905e24c6ef48e0b8a13e0e2
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:42.081472 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.073400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:36341 (local address 127.19.113.126:46823)
0619 02:47:42.073452 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.073455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.073671 (+ 216us) server_negotiation.cc:407] Connection header received
0619 02:47:42.074137 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.074138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.074147 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.074199 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:42.075111 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.075602 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.076723 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.076953 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.077026 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.079068 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:42.079069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:42.079069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:42.079076 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:42.080315 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.080649 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.080650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.080650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.080695 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.080900 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.080901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.080902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.081026 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:42.081090 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.081311 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.081388 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:47:42.081591 1629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.073379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:36341)
0619 02:47:42.073557 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.073594 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.074057 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.074250 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.074253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.074799 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:42.075039 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.075044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.075684 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.075685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.076649 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.076653 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.077039 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.077040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.077186 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.077648 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:42.077657 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:42.078969 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:42.080384 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.080389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.080391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.080595 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.080739 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.080739 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.080739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.080824 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.081083 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:42.081086 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:42.081198 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.081290 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.081405 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260619 02:47:42.082036 1627 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46823
I20260619 02:47:42.082119 1627 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:42.082294 1627 heartbeater.cc:511] Master 127.19.113.126:46823 requested a full tablet report, sending...
I20260619 02:47:42.082638 1168 ts_manager.cc:194] Registered new tserver with Master: 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609)
I20260619 02:47:42.083036 1168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:36341
I20260619 02:47:42.088029 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:47:42.095083 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.089403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.19.113.126:46823)
0619 02:47:42.089464 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.089466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.089565 (+ 99us) server_negotiation.cc:407] Connection header received
0619 02:47:42.089626 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.089626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.089637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.089681 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:42.090223 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.090811 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.091572 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.091820 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.091880 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.093032 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:42.093033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:42.093033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:42.093039 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:42.094217 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.094466 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.094467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.094467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.094531 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.094648 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.094648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.094649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.094776 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:42.094842 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.094957 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.095020 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:47:42.096475 1168 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:47:42.097988 1168 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:47:42.104059 1635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.099934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:42321 (local address 127.0.0.1:43616)
0619 02:47:42.100224 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.100251 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.100307 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.100599 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.100601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.100603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.100651 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.100652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.101801 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.101802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.102937 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.102939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.103762 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.103763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.103843 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.103857 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.103927 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.103980 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":72,"threads_started":1}
I20260619 02:47:42.104072 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.100055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.19.113.66:42321)
0619 02:47:42.100449 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.100453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.100461 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:47:42.100512 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.100513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.100526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.100585 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.100740 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.101719 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.103028 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.103687 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.103757 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.103815 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.103943 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.103983 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":68,"threads_started":1}
I20260619 02:47:42.104439 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.099931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39997 (local address 127.0.0.1:47660)
0619 02:47:42.100209 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.100250 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.100310 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.100917 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.100918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.100920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.100959 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.100961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.102231 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.102233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.103289 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.103291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.104227 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.104228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.104283 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.104286 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.104333 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.104368 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":64,"threads_started":1}
I20260619 02:47:42.104617 1640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.100246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.19.113.67:33609)
0619 02:47:42.100427 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.100431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.101121 (+ 690us) server_negotiation.cc:407] Connection header received
0619 02:47:42.101164 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.101167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.101182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.101260 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.101379 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.102368 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.103586 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.104281 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.104344 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.104400 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.104498 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.104537 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260619 02:47:42.104658 1639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.100134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:33609 (local address 127.0.0.1:58116)
0619 02:47:42.101056 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.101065 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.101107 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.101262 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.101262 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.101263 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.101319 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.101321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.102450 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.102452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.103499 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.103502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.104366 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.104367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.104427 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.104431 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.104485 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.104522 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":54,"threads_started":1}
I20260619 02:47:42.104844 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.100043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47660 (local address 127.19.113.65:39997)
0619 02:47:42.100699 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.100703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.100712 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:47:42.100749 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.100750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.100764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.100833 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.101022 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.102127 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.103370 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.104074 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.104135 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.104192 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.104741 (+ 549us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.104786 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":86,"threads_started":1}
I20260619 02:47:42.105098 1427 tablet_service.cc:1467] Processing CreateTablet for tablet 47990c20b5bd4732b65d3c60b8412294 (DEFAULT_TABLE table=test-table [id=1a01ce16d169461da629328a9492400f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:42.105445 1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47990c20b5bd4732b65d3c60b8412294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:42.105415 1293 tablet_service.cc:1467] Processing CreateTablet for tablet 47990c20b5bd4732b65d3c60b8412294 (DEFAULT_TABLE table=test-table [id=1a01ce16d169461da629328a9492400f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:42.105667 1293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47990c20b5bd4732b65d3c60b8412294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:42.107999 1641 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: Bootstrap starting.
I20260619 02:47:42.108453 1561 tablet_service.cc:1467] Processing CreateTablet for tablet 47990c20b5bd4732b65d3c60b8412294 (DEFAULT_TABLE table=test-table [id=1a01ce16d169461da629328a9492400f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:42.108695 1641 tablet_bootstrap.cc:654] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:42.108781 1561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47990c20b5bd4732b65d3c60b8412294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:42.109326 1641 log.cc:824] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:42.110069 1641 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: No bootstrap required, opened a new log
I20260619 02:47:42.110142 1641 ts_tablet_manager.cc:1397] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:47:42.111107 1642 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Bootstrap starting.
I20260619 02:47:42.111680 1642 tablet_bootstrap.cc:654] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:42.112067 1641 raft_consensus.cc:348] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.112213 1641 raft_consensus.cc:374] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:42.112200 1642 log.cc:824] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:42.112241 1641 raft_consensus.cc:729] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635262b47d2940198774651e5d93abcf, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.112349 1641 consensus_queue.cc:260] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.112629 1493 heartbeater.cc:503] Master 127.19.113.126:46823 was elected leader, sending a full tablet report...
I20260619 02:47:42.112744 1644 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: Bootstrap starting.
I20260619 02:47:42.112615 1641 ts_tablet_manager.cc:1428] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:47:42.112957 1642 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: No bootstrap required, opened a new log
I20260619 02:47:42.113030 1642 ts_tablet_manager.cc:1397] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:47:42.113371 1644 tablet_bootstrap.cc:654] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:42.113937 1644 log.cc:824] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:42.114647 1644 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: No bootstrap required, opened a new log
I20260619 02:47:42.114717 1644 ts_tablet_manager.cc:1397] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:47:42.115136 1642 raft_consensus.cc:348] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.115298 1642 raft_consensus.cc:374] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:42.115335 1642 raft_consensus.cc:729] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97102225dc5f49b599d442616e438c2e, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.115453 1642 consensus_queue.cc:260] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.115727 1642 ts_tablet_manager.cc:1428] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:47:42.115847 1359 heartbeater.cc:503] Master 127.19.113.126:46823 was elected leader, sending a full tablet report...
I20260619 02:47:42.116708 1644 raft_consensus.cc:348] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.116827 1644 raft_consensus.cc:374] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:42.116847 1644 raft_consensus.cc:729] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344aa666d8374552810e4a83c80d5e38, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.116923 1644 consensus_queue.cc:260] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.117120 1644 ts_tablet_manager.cc:1428] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:47:42.117183 1627 heartbeater.cc:503] Master 127.19.113.126:46823 was elected leader, sending a full tablet report...
I20260619 02:47:42.121735 1648 raft_consensus.cc:482] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:42.121829 1648 raft_consensus.cc:504] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.122143 1648 leader_election.cc:288] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 635262b47d2940198774651e5d93abcf (127.19.113.66:42321), 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609)
I20260619 02:47:42.124784 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.122347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:41339 (local address 127.19.113.66:42321)
0619 02:47:42.122395 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.122397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.122424 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:47:42.122452 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.122453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.122464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.122548 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.122684 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.123207 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.124085 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.124467 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.124537 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.124546 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.124681 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.124718 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:47:42.124784 1361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.122282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:42321 (local address 127.19.113.65:41339)
0619 02:47:42.122326 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.122337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.122376 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.122554 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.122556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.122558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.122626 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.122628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.123275 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.123276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.124006 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.124008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.124541 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.124542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.124621 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.124625 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.124669 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.124710 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:47:42.124835 1640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.122459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:36485 (local address 127.19.113.67:33609)
0619 02:47:42.122504 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.122505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.122598 (+ 93us) server_negotiation.cc:407] Connection header received
0619 02:47:42.122651 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.122652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.122661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.122704 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.122829 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.123299 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.124164 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.124529 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.124588 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.124596 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.124737 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.124775 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:47:42.124888 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.122376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:33609 (local address 127.19.113.65:36485)
0619 02:47:42.122552 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.122560 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.122606 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.122705 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.122705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.122706 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.122772 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.122774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.123366 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.123367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.124087 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.124089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.124596 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.124597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.124673 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.124677 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.124721 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.124761 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
I20260619 02:47:42.125152 1447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47990c20b5bd4732b65d3c60b8412294" candidate_uuid: "97102225dc5f49b599d442616e438c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635262b47d2940198774651e5d93abcf" is_pre_election: true
I20260619 02:47:42.125187 1581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47990c20b5bd4732b65d3c60b8412294" candidate_uuid: "97102225dc5f49b599d442616e438c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344aa666d8374552810e4a83c80d5e38" is_pre_election: true
I20260619 02:47:42.125303 1447 raft_consensus.cc:2454] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97102225dc5f49b599d442616e438c2e in term 0.
I20260619 02:47:42.125330 1581 raft_consensus.cc:2454] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97102225dc5f49b599d442616e438c2e in term 0.
I20260619 02:47:42.125507 1246 leader_election.cc:302] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 635262b47d2940198774651e5d93abcf, 97102225dc5f49b599d442616e438c2e; no voters:
I20260619 02:47:42.125634 1648 raft_consensus.cc:2781] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:47:42.125691 1648 raft_consensus.cc:482] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:47:42.125720 1648 raft_consensus.cc:3037] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:42.126391 1648 raft_consensus.cc:504] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.126513 1648 leader_election.cc:288] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 1 election: Requested vote from peers 635262b47d2940198774651e5d93abcf (127.19.113.66:42321), 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609)
I20260619 02:47:42.126677 1581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47990c20b5bd4732b65d3c60b8412294" candidate_uuid: "97102225dc5f49b599d442616e438c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344aa666d8374552810e4a83c80d5e38"
I20260619 02:47:42.126682 1447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47990c20b5bd4732b65d3c60b8412294" candidate_uuid: "97102225dc5f49b599d442616e438c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "635262b47d2940198774651e5d93abcf"
I20260619 02:47:42.126747 1447 raft_consensus.cc:3037] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:42.126748 1581 raft_consensus.cc:3037] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:42.127540 1581 raft_consensus.cc:2454] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97102225dc5f49b599d442616e438c2e in term 1.
I20260619 02:47:42.127562 1447 raft_consensus.cc:2454] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97102225dc5f49b599d442616e438c2e in term 1.
I20260619 02:47:42.127696 1246 leader_election.cc:302] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344aa666d8374552810e4a83c80d5e38, 97102225dc5f49b599d442616e438c2e; no voters:
I20260619 02:47:42.127789 1648 raft_consensus.cc:2781] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:47:42.127938 1648 raft_consensus.cc:686] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 1 LEADER]: Becoming Leader. State: Replica: 97102225dc5f49b599d442616e438c2e, State: Running, Role: LEADER
I20260619 02:47:42.128037 1648 consensus_queue.cc:237] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.128822 1167 catalog_manager.cc:5495] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97102225dc5f49b599d442616e438c2e (127.19.113.65). New cstate: current_term: 1 leader_uuid: "97102225dc5f49b599d442616e438c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } health_report { overall_health: UNKNOWN } } }
I20260619 02:47:42.138799 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.136652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.19.113.65:39997)
0619 02:47:42.136697 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.136699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.136706 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:47:42.136767 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.136768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.136783 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.136839 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:47:42.136983 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.137446 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.138189 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.138396 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.138458 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.138552 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:47:42.138643 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:47:42.138698 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.138712 (+ 14us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.138730 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:47:42.140561 1581 raft_consensus.cc:1261] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 1 FOLLOWER]: Refusing update from remote peer 97102225dc5f49b599d442616e438c2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:47:42.140601 1447 raft_consensus.cc:1261] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 1 FOLLOWER]: Refusing update from remote peer 97102225dc5f49b599d442616e438c2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:47:42.140875 1651 consensus_queue.cc:1035] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:47:42.141018 1648 consensus_queue.cc:1035] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:47:42.143785 1656 mvcc.cc:205] Tried to move back new op lower bound from 7298405425725046784 to 7298405425676836864. Current Snapshot: MvccSnapshot[applied={T|T < 7298405425725046784}]
I20260619 02:47:42.144243 1655 mvcc.cc:205] Tried to move back new op lower bound from 7298405425725046784 to 7298405425676836864. Current Snapshot: MvccSnapshot[applied={T|T < 7298405425725046784}]
I20260619 02:47:42.145601 1168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:47:42.145681 1168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:47:42.146329 1168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Sending DeleteTablet for 3 replicas of tablet 47990c20b5bd4732b65d3c60b8412294
I20260619 02:47:42.146641 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 1230
I20260619 02:47:42.146629 1293 tablet_service.cc:1514] Processing DeleteTablet for tablet 47990c20b5bd4732b65d3c60b8412294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:42 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:47660
I20260619 02:47:42.146701 1427 tablet_service.cc:1514] Processing DeleteTablet for tablet 47990c20b5bd4732b65d3c60b8412294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:42 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:43616
I20260619 02:47:42.146885 1561 tablet_service.cc:1514] Processing DeleteTablet for tablet 47990c20b5bd4732b65d3c60b8412294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:42 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:58116
I20260619 02:47:42.147324 1665 tablet_replica.cc:330] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: stopping tablet replica
I20260619 02:47:42.147415 1665 raft_consensus.cc:2229] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:47:42.147521 1665 raft_consensus.cc:2258] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:47:42.147670 1664 tablet_replica.cc:330] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: stopping tablet replica
I20260619 02:47:42.147763 1664 raft_consensus.cc:2229] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:47:42.147857 1664 raft_consensus.cc:2258] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:47:42.148113 1665 ts_tablet_manager.cc:1905] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:47:42.148198 1664 ts_tablet_manager.cc:1905] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:47:42.149897 1665 ts_tablet_manager.cc:1918] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:47:42.149897 1664 ts_tablet_manager.cc:1918] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:47:42.149971 1665 log.cc:1196] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/47990c20b5bd4732b65d3c60b8412294
I20260619 02:47:42.149971 1664 log.cc:1196] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/47990c20b5bd4732b65d3c60b8412294
I20260619 02:47:42.150240 1665 ts_tablet_manager.cc:1939] T 47990c20b5bd4732b65d3c60b8412294 P 344aa666d8374552810e4a83c80d5e38: Deleting consensus metadata
I20260619 02:47:42.150240 1664 ts_tablet_manager.cc:1939] T 47990c20b5bd4732b65d3c60b8412294 P 635262b47d2940198774651e5d93abcf: Deleting consensus metadata
I20260619 02:47:42.150624 1152 catalog_manager.cc:4841] TS 635262b47d2940198774651e5d93abcf (127.19.113.66:42321): tablet 47990c20b5bd4732b65d3c60b8412294 (table test-table [id=1a01ce16d169461da629328a9492400f]) successfully deleted
I20260619 02:47:42.150729 1152 catalog_manager.cc:4841] TS 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609): tablet 47990c20b5bd4732b65d3c60b8412294 (table test-table [id=1a01ce16d169461da629328a9492400f]) successfully deleted
W20260619 02:47:42.151516 1153 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:39997 (error 108)
I20260619 02:47:42.151660 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 1364
I20260619 02:47:42.152552 1635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.152441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39997 (local address 127.0.0.1:47668)
0619 02:47:42.152483 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.152525 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 02:47:42.152667 1153 catalog_manager.cc:4584] TS 97102225dc5f49b599d442616e438c2e (127.19.113.65:39997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47990c20b5bd4732b65d3c60b8412294: Network error: Client connection negotiation failed: client connection to 127.19.113.65:39997: connect: Connection refused (error 111)
I20260619 02:47:42.156013 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 1498
I20260619 02:47:42.159878 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 1136
I20260619 02:47:42.164121 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46823
--webserver_interface=127.19.113.126
--webserver_port=37273
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:42.243193 1667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:42.243403 1667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:42.243433 1667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:42.244618 1667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:42.244663 1667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:42.244675 1667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:42.244688 1667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:42.244710 1667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:42.246222 1667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:46823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=37273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:42.246469 1667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:42.246696 1667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:42.254793 1676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.254793 1673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.255014 1674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:42.254905 1667 server_base.cc:1034] running on GCE node
I20260619 02:47:42.255246 1667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:42.255511 1667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:42.256644 1667 hybrid_clock.cc:648] HybridClock initialized: now 1781837262256625 us; error 34 us; skew 500 ppm
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:42.259021 1667 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:47:42.259428 1667 webserver.cc:460] Webserver started at http://127.19.113.126:37273/ using document root <none> and password file <none>
I20260619 02:47:42.259640 1667 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:42.259696 1667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:42.261009 1667 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:42.261579 1683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.261720 1667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:42.261802 1667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "0d128cf3bf674b5fafb7a6897e745a4a"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "e04016f60f034d1510d0795994636bd7"
server_key_iv: "65b023bcc802be647f3e8b744f299798"
server_key_version: "encryptionkey@0"
I20260619 02:47:42.262207 1667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:42.283571 1667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:42.285615 1667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:42.285723 1667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:42.289479 1667 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:46823
I20260619 02:47:42.289512 1735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:46823 every 8 connection(s)
I20260619 02:47:42.289872 1667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:47:42.291533 1736 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:47:42.292147 1736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Bootstrap starting.
I20260619 02:47:42.298732 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 1667
I20260619 02:47:42.299070 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:39997
--local_ip_for_outbound_sockets=127.19.113.65
--tserver_master_addrs=127.19.113.126:46823
--webserver_port=44685
--webserver_interface=127.19.113.65
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:47:42.301076 1736 log.cc:824] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:42.303033 1736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:47:42.303306 1736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Bootstrap complete.
I20260619 02:47:42.305385 1736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:42.305605 1736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d128cf3bf674b5fafb7a6897e745a4a, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.305745 1736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:42.305804 1736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:47:42.305848 1736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:47:42.305893 1736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:47:42.306928 1736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:42.307024 1736 leader_election.cc:302] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d128cf3bf674b5fafb7a6897e745a4a; no voters:
I20260619 02:47:42.307189 1736 leader_election.cc:288] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:47:42.307238 1740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:47:42.307348 1740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 2 LEADER]: Becoming Leader. State: Replica: 0d128cf3bf674b5fafb7a6897e745a4a, State: Running, Role: LEADER
I20260619 02:47:42.307433 1740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:42.307435 1736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:42.307716 1741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } } }
I20260619 02:47:42.307789 1741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:42.307926 1742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d128cf3bf674b5fafb7a6897e745a4a. Latest consensus state: current_term: 2 leader_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } } }
I20260619 02:47:42.307973 1742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:42.308336 1744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:47:42.309152 1744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:47:42.309378 1744 catalog_manager.cc:1259] Loaded cluster ID: 23ead91f36ce4d6c886cdde9746b4c17
I20260619 02:47:42.309514 1744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:47:42.309749 1744 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 02:47:42.314360 1744 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 02:47:42.316821 1744 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:47:42.317021 1744 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Loaded TSK: 0
I20260619 02:47:42.317257 1744 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:47:42.381361 1738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:42.381578 1738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:42.381610 1738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:42.382807 1738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:42.382850 1738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:42.382920 1738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:47:42.384438 1738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:39997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=44685
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:42.384686 1738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:42.384896 1738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:42.389806 1766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:42.389832 1738 server_base.cc:1034] running on GCE node
W20260619 02:47:42.389811 1763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.390111 1764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:42.390318 1738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:42.390568 1738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:42.391705 1738 hybrid_clock.cc:648] HybridClock initialized: now 1781837262391695 us; error 35 us; skew 500 ppm
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:42.394052 1738 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:47:42.394476 1738 webserver.cc:460] Webserver started at http://127.19.113.65:44685/ using document root <none> and password file <none>
I20260619 02:47:42.394697 1738 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:42.394759 1738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:42.396129 1738 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:42.396791 1773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.396961 1738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:42.397042 1738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "97102225dc5f49b599d442616e438c2e"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "7ee1631d2f82a77cfe56fd64f66f8c1b"
server_key_iv: "d7888b74df2fc31411281941d2911977"
server_key_version: "encryptionkey@0"
I20260619 02:47:42.397398 1738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:42.417788 1738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:42.419909 1738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:42.420055 1738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:42.420281 1738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:42.420701 1780 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:47:42.422576 1738 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:47:42.422634 1738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:47:42.422672 1738 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:47:42.423354 1738 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:47:42.423388 1738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:47:42.423473 1780 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Bootstrap starting.
I20260619 02:47:42.429934 1738 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:39997
I20260619 02:47:42.430114 1887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:39997 every 8 connection(s)
I20260619 02:47:42.430366 1738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:47:42.434343 1780 log.cc:824] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:42.434369 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 1738
I20260619 02:47:42.434613 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:42321
--local_ip_for_outbound_sockets=127.19.113.66
--tserver_master_addrs=127.19.113.126:46823
--webserver_port=36687
--webserver_interface=127.19.113.66
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:47:42.437297 1780 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:47:42.437605 1780 tablet_bootstrap.cc:492] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Bootstrap complete.
I20260619 02:47:42.438334 1780 ts_tablet_manager.cc:1397] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20260619 02:47:42.439440 1780 raft_consensus.cc:348] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.439677 1780 raft_consensus.cc:729] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97102225dc5f49b599d442616e438c2e, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.439831 1780 consensus_queue.cc:260] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } } peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } }
I20260619 02:47:42.440084 1780 ts_tablet_manager.cc:1428] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:42.446187 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.431141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:55583 (local address 127.19.113.126:46823)
0619 02:47:42.435164 (+ 4023us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.435173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.435216 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:47:42.436497 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.436503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.436567 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.436697 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:42.437453 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.438723 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.440238 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.440632 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.440711 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.443102 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:42.443106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:42.443106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:42.443116 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:42.444549 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.444996 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.444999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.444999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.445082 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.445342 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.445343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.445343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.445563 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:42.445620 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.445915 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.446018 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3970,"thread_start_us":72,"threads_started":1}
I20260619 02:47:42.446244 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.431063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:55583)
0619 02:47:42.434638 (+ 3575us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.434682 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.436211 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.436833 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.436837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.437118 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:42.437305 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.437310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.438837 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.438838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.440134 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.440138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.440718 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.440719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.440864 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.441380 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:42.441390 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:42.442996 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:42.444629 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.444636 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.444638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.444920 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.445140 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.445141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.445141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.445283 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.445619 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:42.445623 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:42.445768 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.445866 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.446035 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3532,"thread_start_us":93,"threads_started":1}
I20260619 02:47:42.446935 1888 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46823
I20260619 02:47:42.447028 1888 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:42.447191 1888 heartbeater.cc:511] Master 127.19.113.126:46823 requested a full tablet report, sending...
I20260619 02:47:42.447849 1700 ts_manager.cc:194] Registered new tserver with Master: 97102225dc5f49b599d442616e438c2e (127.19.113.65:39997)
I20260619 02:47:42.449047 1700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:55583
I20260619 02:47:42.449949 1888 heartbeater.cc:503] Master 127.19.113.126:46823 was elected leader, sending a full tablet report...
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:47:42.455693 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.448425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39997 (local address 127.0.0.1:47682)
0619 02:47:42.448706 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.448732 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.448809 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.449026 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.449031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.449328 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:42.449391 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.449394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.450301 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.450302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.450967 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.450970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.451413 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.451414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.451501 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.451904 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:42.451909 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:42.453102 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:42.454618 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.454625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.454627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.454812 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.455031 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.455032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.455032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.455092 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.455336 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:42.455343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:42.455395 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.455468 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.455597 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":113,"threads_started":1}
I20260619 02:47:42.455693 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.448495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.19.113.65:39997)
0619 02:47:42.448648 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.448652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.448809 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:47:42.448882 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.448884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.448898 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.448958 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:42.449465 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.450215 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.451038 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.451348 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.451403 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.453179 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:42.453186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:42.453187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:42.453193 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:42.454536 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.454906 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.454909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.454910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.454968 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.455141 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.455142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.455142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.455277 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:42.455347 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.455510 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.455595 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260619 02:47:42.456059 1822 tablet_service.cc:1514] Processing DeleteTablet for tablet 47990c20b5bd4732b65d3c60b8412294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:42 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:47682
I20260619 02:47:42.456537 1897 tablet_replica.cc:330] stopping tablet replica
I20260619 02:47:42.456646 1897 raft_consensus.cc:2229] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:47:42.456710 1897 raft_consensus.cc:2258] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:47:42.457010 1897 ts_tablet_manager.cc:1905] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:47:42.459237 1897 ts_tablet_manager.cc:1918] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:47:42.459316 1897 log.cc:1196] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/47990c20b5bd4732b65d3c60b8412294
I20260619 02:47:42.459522 1897 ts_tablet_manager.cc:1939] T 47990c20b5bd4732b65d3c60b8412294 P 97102225dc5f49b599d442616e438c2e: Deleting consensus metadata
I20260619 02:47:42.460024 1685 catalog_manager.cc:4841] TS 97102225dc5f49b599d442616e438c2e (127.19.113.65:39997): tablet 47990c20b5bd4732b65d3c60b8412294 (table test-table [id=1a01ce16d169461da629328a9492400f]) successfully deleted
W20260619 02:47:42.519675 1893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:42.519881 1893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:42.519904 1893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:42.521077 1893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:42.521119 1893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:42.521147 1893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:47:42.522665 1893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:42321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=36687
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:42.522919 1893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:42.523139 1893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:42.527948 1903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.527948 1902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.527946 1905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:42.528004 1893 server_base.cc:1034] running on GCE node
I20260619 02:47:42.528318 1893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:42.528532 1893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:42.529682 1893 hybrid_clock.cc:648] HybridClock initialized: now 1781837262529666 us; error 27 us; skew 500 ppm
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:42.532119 1893 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:47:42.532506 1893 webserver.cc:460] Webserver started at http://127.19.113.66:36687/ using document root <none> and password file <none>
I20260619 02:47:42.532696 1893 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:42.532747 1893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:42.534010 1893 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:42.534569 1912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.534714 1893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:42.534783 1893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "635262b47d2940198774651e5d93abcf"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "678ac7be66d0c4187e0e8e256ea55746"
server_key_iv: "88013af157c6f53434e239159b27fd19"
server_key_version: "encryptionkey@0"
I20260619 02:47:42.535177 1893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:42.547441 1893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:42.549525 1893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:42.549644 1893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:42.549849 1893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:42.550128 1893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:42.550159 1893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.550180 1893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:42.550204 1893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.555475 1893 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:42321
I20260619 02:47:42.555521 2025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:42321 every 8 connection(s)
I20260619 02:47:42.555821 1893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:47:42.560402 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 1893
I20260619 02:47:42.560633 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:33609
--local_ip_for_outbound_sockets=127.19.113.67
--tserver_master_addrs=127.19.113.126:46823
--webserver_port=33777
--webserver_interface=127.19.113.67
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:42.564563 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.556511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:59693 (local address 127.19.113.126:46823)
0619 02:47:42.556560 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.556563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.556773 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:47:42.557268 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.557270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.557284 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.557336 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:42.558202 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.558732 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.559705 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.559909 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.559955 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.561939 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:42.561941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:42.561941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:42.561950 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:42.563315 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.563663 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.563664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.563664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.563724 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.563913 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.563914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.563915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.564066 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:42.564136 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.564375 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.564468 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:47:42.564667 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.556503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:59693)
0619 02:47:42.556662 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.556696 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.557165 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.557397 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.557400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.557941 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:42.558128 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.558132 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.558800 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.558801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.559641 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.559644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.559962 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.559963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.560056 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.560508 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:42.560519 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:42.561845 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:42.563385 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.563390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.563392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.563599 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.563779 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.563779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.563779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.563860 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.564124 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:42.564129 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:42.564241 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.564338 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.564489 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260619 02:47:42.565106 2026 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46823
I20260619 02:47:42.565224 2026 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:42.565405 2026 heartbeater.cc:511] Master 127.19.113.126:46823 requested a full tablet report, sending...
I20260619 02:47:42.565783 1700 ts_manager.cc:194] Registered new tserver with Master: 635262b47d2940198774651e5d93abcf (127.19.113.66:42321)
I20260619 02:47:42.566280 1700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:59693
W20260619 02:47:42.640833 2029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:42.641032 2029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:42.641055 2029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:42.642186 2029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:42.642226 2029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:42.642256 2029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:47:42.643728 2029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:33609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=33777
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:42.643939 2029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:42.644142 2029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:42.648676 2037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.648706 2035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:42.648770 2029 server_base.cc:1034] running on GCE node
W20260619 02:47:42.648924 2034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:42.649122 2029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:42.649341 2029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:42.650482 2029 hybrid_clock.cc:648] HybridClock initialized: now 1781837262650469 us; error 27 us; skew 500 ppm
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:42.667101 2029 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:47:42.667551 2029 webserver.cc:460] Webserver started at http://127.19.113.67:33777/ using document root <none> and password file <none>
I20260619 02:47:42.667750 2029 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:42.667798 2029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:42.669046 2029 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:47:42.669567 2044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.669754 2029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.669828 2029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "344aa666d8374552810e4a83c80d5e38"
format_stamp: "Formatted at 2026-06-19 02:47:42 on dist-test-slave-zqpb"
server_key: "42567174a32fc86644dea421a039cac8"
server_key_iv: "442e8ae62ec257db258feaf2f9eb1df5"
server_key_version: "encryptionkey@0"
I20260619 02:47:42.670138 2029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:42.679327 2029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:42.680946 2029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:42.681052 2029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:42.681243 2029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:42.681514 2029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:42.681546 2029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.681567 2029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:42.681591 2029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.686841 2029 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:33609
I20260619 02:47:42.686926 2157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:33609 every 8 connection(s)
I20260619 02:47:42.687243 2029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:42.695647 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 2029
I20260619 02:47:42.696893 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.687888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:39371 (local address 127.19.113.126:46823)
0619 02:47:42.687933 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.687935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.688166 (+ 231us) server_negotiation.cc:407] Connection header received
0619 02:47:42.688702 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.688703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.688711 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.688761 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:42.689740 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.690223 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.691219 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.691401 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.691451 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.693893 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:42.693895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:42.693895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:42.693904 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:42.695423 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.695932 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.695933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.695933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.696005 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.696260 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.696261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.696262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.696399 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:42.696463 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.696717 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.696804 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:47:42.696954 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.687874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:39371)
0619 02:47:42.688044 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.688084 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.688626 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.688832 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.688835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.689420 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:42.689658 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.689664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.690282 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.690283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.691156 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.691159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.691457 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.691458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.691555 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.692071 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:42.692082 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:42.693794 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:42.695522 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.695529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.695531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.695826 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.696070 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:42.696070 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:42.696071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:42.696195 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:42.696465 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:42.696470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:42.696592 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.696672 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.696792 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260619 02:47:42.698141 2158 heartbeater.cc:347] Connected to a master server at 127.19.113.126:46823
I20260619 02:47:42.698243 2158 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:42.698429 2158 heartbeater.cc:511] Master 127.19.113.126:46823 requested a full tablet report, sending...
I20260619 02:47:42.699484 1700 ts_manager.cc:194] Registered new tserver with Master: 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609)
I20260619 02:47:42.705302 1700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:39371
I20260619 02:47:42.709918 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.697271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.19.113.126:46823)
0619 02:47:42.697482 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.697484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.697583 (+ 99us) server_negotiation.cc:407] Connection header received
0619 02:47:42.697627 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.697628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.697638 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.697693 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:42.698608 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.699298 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.705104 (+ 5806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.705396 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.705469 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.707628 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:42.707629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:42.707630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:42.707637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:42.708929 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.709238 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.709239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.709240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.709293 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:42.709451 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:42.709452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:42.709452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:42.709562 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:42.709724 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.709759 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.709830 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":193}
I20260619 02:47:42.710619 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:47:42.713191 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.710933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54758 (local address 127.19.113.126:46823)
0619 02:47:42.710974 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.710976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.711097 (+ 121us) server_negotiation.cc:407] Connection header received
0619 02:47:42.711158 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.711159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.711198 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.711256 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:47:42.711349 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.711813 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.712534 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.712728 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.712779 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.712913 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:47:42.713011 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:47:42.713079 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.713113 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.713130 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:47:42.713510 1700 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:47:42.715070 1700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:47:42.716895 1822 tablet_service.cc:1467] Processing CreateTablet for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c (DEFAULT_TABLE table=test-table [id=e1b8a194686f4019b1a7de5d65bfd34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:42.717085 1822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:42.718608 1780 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Bootstrap starting.
I20260619 02:47:42.718912 1780 tablet_bootstrap.cc:654] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:42.719666 1780 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: No bootstrap required, opened a new log
I20260619 02:47:42.719709 1780 ts_tablet_manager.cc:1397] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:47:42.719852 1780 raft_consensus.cc:348] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.719915 1780 raft_consensus.cc:374] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:42.719961 1780 raft_consensus.cc:729] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97102225dc5f49b599d442616e438c2e, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.720011 1780 consensus_queue.cc:260] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.720103 1780 ts_tablet_manager.cc:1428] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.720763 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.716640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:33609 (local address 127.0.0.1:58126)
0619 02:47:42.716815 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.716825 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.716858 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.717097 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.717098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.717101 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.717142 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.717143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.718447 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.718448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.719667 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.719670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.720524 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.720525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.720595 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.720619 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.720648 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.720694 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":24}
I20260619 02:47:42.721015 2163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.716726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:42321 (local address 127.0.0.1:43630)
0619 02:47:42.716821 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.716831 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.716878 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.717371 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.717373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.717375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.717418 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.717419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.718381 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.718382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.719944 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.719947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.720778 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.720778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.720840 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.720844 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.720896 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.720935 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":34,"thread_start_us":56,"threads_started":1}
I20260619 02:47:42.721029 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.717008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43630 (local address 127.19.113.66:42321)
0619 02:47:42.717229 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.717234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.717243 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:47:42.717277 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.717279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.717293 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.717361 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.717467 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.718296 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.720025 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.720716 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.720779 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.720836 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.720909 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.720950 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
I20260619 02:47:42.721123 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.716732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.19.113.67:33609)
0619 02:47:42.716932 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.716936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.716945 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:47:42.716987 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.716989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.717003 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.717081 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.717225 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.718322 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.719750 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.720435 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.720922 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.720990 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.721012 (+ 22us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.721050 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":100,"threads_started":1}
I20260619 02:47:42.721626 1960 tablet_service.cc:1467] Processing CreateTablet for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c (DEFAULT_TABLE table=test-table [id=e1b8a194686f4019b1a7de5d65bfd34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:42.721623 2092 tablet_service.cc:1467] Processing CreateTablet for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c (DEFAULT_TABLE table=test-table [id=e1b8a194686f4019b1a7de5d65bfd34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:47:42.721903 1960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:42.721923 2092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:42.724423 2167 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: Bootstrap starting.
I20260619 02:47:42.724611 2168 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Bootstrap starting.
I20260619 02:47:42.725045 2167 tablet_bootstrap.cc:654] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:42.725204 2168 tablet_bootstrap.cc:654] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:42.725459 2167 log.cc:824] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:42.725682 2168 log.cc:824] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:42.726117 2167 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: No bootstrap required, opened a new log
I20260619 02:47:42.726174 2167 ts_tablet_manager.cc:1397] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:47:42.726341 2168 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: No bootstrap required, opened a new log
I20260619 02:47:42.726397 2168 ts_tablet_manager.cc:1397] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:47:42.727488 2167 raft_consensus.cc:348] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.727615 2167 raft_consensus.cc:374] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:42.727636 2167 raft_consensus.cc:729] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344aa666d8374552810e4a83c80d5e38, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.727766 2167 consensus_queue.cc:260] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.727798 2168 raft_consensus.cc:348] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.727948 2168 raft_consensus.cc:374] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:42.727988 2168 raft_consensus.cc:729] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635262b47d2940198774651e5d93abcf, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.728021 2158 heartbeater.cc:503] Master 127.19.113.126:46823 was elected leader, sending a full tablet report...
I20260619 02:47:42.728117 2168 consensus_queue.cc:260] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.728384 2167 ts_tablet_manager.cc:1428] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:47:42.728379 2026 heartbeater.cc:503] Master 127.19.113.126:46823 was elected leader, sending a full tablet report...
I20260619 02:47:42.728379 2168 ts_tablet_manager.cc:1428] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:47:42.787600 2172 raft_consensus.cc:482] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:42.787714 2172 raft_consensus.cc:504] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.788087 2172 leader_election.cc:288] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609), 97102225dc5f49b599d442616e438c2e (127.19.113.65:39997)
I20260619 02:47:42.791100 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.788468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:36815 (local address 127.19.113.67:33609)
0619 02:47:42.788523 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.788525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.788574 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:47:42.788666 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.788667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.788675 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.788727 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.788930 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.789459 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.790402 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.790766 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.790816 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.790823 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.790993 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.791033 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:47:42.791143 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.788344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:49677 (local address 127.19.113.65:39997)
0619 02:47:42.788383 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.788385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.788397 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:47:42.788460 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.788461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.788489 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.788578 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:47:42.788753 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.789409 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.790403 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.790797 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.790888 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.790966 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.791039 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.791083 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:47:42.791133 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.788274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:33609 (local address 127.19.113.66:36815)
0619 02:47:42.788506 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.788515 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.788605 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.788779 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.788780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.788782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.788854 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.788856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.789536 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.789537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.790317 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.790320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.790841 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.790842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.790927 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.790931 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.790980 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.791019 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":20,"thread_start_us":94,"threads_started":1}
I20260619 02:47:42.791136 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.788272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39997 (local address 127.19.113.66:49677)
0619 02:47:42.788350 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.788361 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:42.788401 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:42.788601 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:42.788602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:42.788604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:47:42.788675 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.788678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.789484 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.789485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.790310 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:42.790313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.790893 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.790894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:42.790972 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.790975 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:47:42.791026 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:47:42.791060 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":51}
I20260619 02:47:42.791705 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f57bcdc2d0c40b5a249ee45350d5b0c" candidate_uuid: "635262b47d2940198774651e5d93abcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344aa666d8374552810e4a83c80d5e38" is_pre_election: true
I20260619 02:47:42.791690 1842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f57bcdc2d0c40b5a249ee45350d5b0c" candidate_uuid: "635262b47d2940198774651e5d93abcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97102225dc5f49b599d442616e438c2e" is_pre_election: true
I20260619 02:47:42.791831 1842 raft_consensus.cc:2454] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 635262b47d2940198774651e5d93abcf in term 0.
I20260619 02:47:42.791838 2112 raft_consensus.cc:2454] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 635262b47d2940198774651e5d93abcf in term 0.
I20260619 02:47:42.792033 1913 leader_election.cc:302] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344aa666d8374552810e4a83c80d5e38, 635262b47d2940198774651e5d93abcf; no voters:
I20260619 02:47:42.792160 2172 raft_consensus.cc:2781] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:47:42.792205 2172 raft_consensus.cc:482] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:47:42.792220 2172 raft_consensus.cc:3037] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:42.792889 2172 raft_consensus.cc:504] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.793010 2172 leader_election.cc:288] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [CANDIDATE]: Term 1 election: Requested vote from peers 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609), 97102225dc5f49b599d442616e438c2e (127.19.113.65:39997)
I20260619 02:47:42.793133 1842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f57bcdc2d0c40b5a249ee45350d5b0c" candidate_uuid: "635262b47d2940198774651e5d93abcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97102225dc5f49b599d442616e438c2e"
I20260619 02:47:42.793154 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f57bcdc2d0c40b5a249ee45350d5b0c" candidate_uuid: "635262b47d2940198774651e5d93abcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344aa666d8374552810e4a83c80d5e38"
I20260619 02:47:42.793211 1842 raft_consensus.cc:3037] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:42.793223 2112 raft_consensus.cc:3037] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:42.793912 1842 raft_consensus.cc:2454] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 635262b47d2940198774651e5d93abcf in term 1.
I20260619 02:47:42.793934 2112 raft_consensus.cc:2454] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 635262b47d2940198774651e5d93abcf in term 1.
I20260619 02:47:42.794080 1914 leader_election.cc:302] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 635262b47d2940198774651e5d93abcf, 97102225dc5f49b599d442616e438c2e; no voters:
I20260619 02:47:42.794164 2172 raft_consensus.cc:2781] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:47:42.794302 2172 raft_consensus.cc:686] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 1 LEADER]: Becoming Leader. State: Replica: 635262b47d2940198774651e5d93abcf, State: Running, Role: LEADER
I20260619 02:47:42.794384 2172 consensus_queue.cc:237] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:42.795152 1699 catalog_manager.cc:5495] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 635262b47d2940198774651e5d93abcf (127.19.113.66). New cstate: current_term: 1 leader_uuid: "635262b47d2940198774651e5d93abcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } health_report { overall_health: UNKNOWN } } }
W20260619 02:47:42.806348 2027 tablet.cc:2369] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:47:42.811879 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.809622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.19.113.66:42321)
0619 02:47:42.809666 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:42.809668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:42.809677 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:47:42.809755 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:42.809756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:42.809773 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:42.809829 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:47:42.809986 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.810445 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.811199 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:42.811400 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:42.811453 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:42.811590 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:47:42.811681 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:47:42.811748 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:42.811768 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 02:47:42.811787 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:47:42.813530 1842 raft_consensus.cc:1261] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Refusing update from remote peer 635262b47d2940198774651e5d93abcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:47:42.813530 2112 raft_consensus.cc:1261] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 1 FOLLOWER]: Refusing update from remote peer 635262b47d2940198774651e5d93abcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:47:42.813867 2172 consensus_queue.cc:1035] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:47:42.814002 2174 consensus_queue.cc:1035] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:47:42.816291 2178 mvcc.cc:205] Tried to move back new op lower bound from 7298405428481638400 to 7298405428406353920. Current Snapshot: MvccSnapshot[applied={T|T < 7298405428481638400}]
I20260619 02:47:42.816990 2179 mvcc.cc:205] Tried to move back new op lower bound from 7298405428481638400 to 7298405428406353920. Current Snapshot: MvccSnapshot[applied={T|T < 7298405428481638400 or (T in {7298405428481638400})}]
I20260619 02:47:42.818856 1699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:47:42.818989 1699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:47:42.819658 1699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Sending DeleteTablet for 3 replicas of tablet 6f57bcdc2d0c40b5a249ee45350d5b0c
I20260619 02:47:42.819944 1822 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:42 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:47682
I20260619 02:47:42.819939 2092 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:42 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:58126
I20260619 02:47:42.819962 1960 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:47:42 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:43630
I20260619 02:47:42.820104 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 1738
I20260619 02:47:42.820143 1897 tablet_replica.cc:330] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: stopping tablet replica
I20260619 02:47:42.820182 2186 tablet_replica.cc:330] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: stopping tablet replica
I20260619 02:47:42.820194 1897 raft_consensus.cc:2229] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:47:42.820245 1897 raft_consensus.cc:2258] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:47:42.820272 2186 raft_consensus.cc:2229] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:47:42.820358 2186 raft_consensus.cc:2258] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:47:42.820487 2187 tablet_replica.cc:330] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: stopping tablet replica
I20260619 02:47:42.820562 2187 raft_consensus.cc:2229] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:47:42.820660 2187 raft_consensus.cc:2258] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:47:42.821062 2187 ts_tablet_manager.cc:1905] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:47:42.822119 2186 ts_tablet_manager.cc:1905] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:47:42.822856 2187 ts_tablet_manager.cc:1918] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:47:42.822939 2187 log.cc:1196] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/6f57bcdc2d0c40b5a249ee45350d5b0c
I20260619 02:47:42.823189 2187 ts_tablet_manager.cc:1939] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 635262b47d2940198774651e5d93abcf: Deleting consensus metadata
I20260619 02:47:42.823544 1684 catalog_manager.cc:4841] TS 635262b47d2940198774651e5d93abcf (127.19.113.66:42321): tablet 6f57bcdc2d0c40b5a249ee45350d5b0c (table test-table [id=e1b8a194686f4019b1a7de5d65bfd34f]) successfully deleted
I20260619 02:47:42.823750 2186 ts_tablet_manager.cc:1918] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:47:42.823808 2186 log.cc:1196] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/6f57bcdc2d0c40b5a249ee45350d5b0c
I20260619 02:47:42.824074 2186 ts_tablet_manager.cc:1939] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 344aa666d8374552810e4a83c80d5e38: Deleting consensus metadata
I20260619 02:47:42.824540 1684 catalog_manager.cc:4841] TS 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609): tablet 6f57bcdc2d0c40b5a249ee45350d5b0c (table test-table [id=e1b8a194686f4019b1a7de5d65bfd34f]) successfully deleted
W20260619 02:47:42.825322 1685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:39997 (error 108)
I20260619 02:47:42.825716 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 1893
I20260619 02:47:42.825855 2163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:42.825727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:39997 (local address 127.0.0.1:47692)
0619 02:47:42.825784 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:42.825822 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:47:42.827015 1685 catalog_manager.cc:4584] TS 97102225dc5f49b599d442616e438c2e (127.19.113.65:39997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f57bcdc2d0c40b5a249ee45350d5b0c: Network error: Client connection negotiation failed: client connection to 127.19.113.65:39997: connect: Connection refused (error 111)
I20260619 02:47:42.829725 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 2029
I20260619 02:47:42.833770 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 1667
I20260619 02:47:42.838110 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46823
--webserver_interface=127.19.113.126
--webserver_port=37273
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:42.916479 2189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:42.916684 2189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:42.916706 2189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:42.917870 2189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:42.917913 2189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:42.917927 2189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:42.917938 2189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:42.917948 2189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:42.919448 2189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:46823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=37273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:42.919672 2189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:42.919890 2189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:47:42.924458 2189 server_base.cc:1034] running on GCE node
W20260619 02:47:42.924401 2197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.924424 2194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:42.924391 2195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:42.925168 2189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:42.925448 2189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:42.926589 2189 hybrid_clock.cc:648] HybridClock initialized: now 1781837262926573 us; error 32 us; skew 500 ppm
Jun 19 02:47:42 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:42.928869 2189 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:47:42.929271 2189 webserver.cc:460] Webserver started at http://127.19.113.126:37273/ using document root <none> and password file <none>
I20260619 02:47:42.929482 2189 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:42.929530 2189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:42.930850 2189 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:47:42.931511 2204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:42.931723 2189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:42.931871 2189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "0d128cf3bf674b5fafb7a6897e745a4a"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "e04016f60f034d1510d0795994636bd7"
server_key_iv: "65b023bcc802be647f3e8b744f299798"
server_key_version: "encryptionkey@0"
I20260619 02:47:42.932215 2189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:42.942042 2189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:42.944182 2189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:42.944311 2189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:42.947974 2189 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:46823
I20260619 02:47:42.948007 2256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:46823 every 8 connection(s)
I20260619 02:47:42.948334 2189 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:47:42.949913 2257 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:47:42.950480 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Bootstrap starting.
I20260619 02:47:42.952560 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 2189
I20260619 02:47:42.952847 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:39997
--local_ip_for_outbound_sockets=127.19.113.65
--tserver_master_addrs=127.19.113.126:46823
--webserver_port=44685
--webserver_interface=127.19.113.65
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:47:42.958959 2257 log.cc:824] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:42.961064 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:47:42.961338 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a: Bootstrap complete.
I20260619 02:47:42.963449 2257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:42.963678 2257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d128cf3bf674b5fafb7a6897e745a4a, State: Initialized, Role: FOLLOWER
I20260619 02:47:42.963835 2257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:42.963892 2257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:47:42.963932 2257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:47:42.963976 2257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:47:42.964952 2257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:42.965041 2257 leader_election.cc:302] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d128cf3bf674b5fafb7a6897e745a4a; no voters:
I20260619 02:47:42.965193 2257 leader_election.cc:288] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:47:42.965238 2261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:47:42.965363 2261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [term 3 LEADER]: Becoming Leader. State: Replica: 0d128cf3bf674b5fafb7a6897e745a4a, State: Running, Role: LEADER
I20260619 02:47:42.965447 2261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } }
I20260619 02:47:42.965452 2257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:42.966116 2262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d128cf3bf674b5fafb7a6897e745a4a. Latest consensus state: current_term: 3 leader_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } } }
I20260619 02:47:42.966126 2263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d128cf3bf674b5fafb7a6897e745a4a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 46823 } } }
I20260619 02:47:42.966207 2262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:42.966231 2263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d128cf3bf674b5fafb7a6897e745a4a [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:42.966480 2271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:47:42.967253 2271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:47:42.967361 2271 catalog_manager.cc:1259] Loaded cluster ID: 23ead91f36ce4d6c886cdde9746b4c17
I20260619 02:47:42.967384 2271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 02:47:42.967586 2271 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7faf132279ad google::LogMessage::Fail() at ??:0
@ 0x7faf1322a88a google::LogMessage::SendToLog() at ??:0
@ 0x7faf132274de google::LogMessage::Flush() at ??:0
@ 0x7faf13228a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7faf1b1480f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7faf1b1518b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7faf150f50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7faf150ea954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7faf15f4e6db start_thread at ??:0
@ 0x7faf11d2561f clone at ??:0
W20260619 02:47:43.044118 2259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:43.044317 2259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:43.044341 2259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:43.045518 2259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:43.045559 2259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:43.045586 2259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:47:43.047099 2259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:39997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=44685
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:43.047317 2259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:43.047542 2259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:43.052436 2293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:43.052398 2294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:43.052490 2259 server_base.cc:1034] running on GCE node
W20260619 02:47:43.052677 2296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:43.052913 2259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:43.053139 2259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:43.054291 2259 hybrid_clock.cc:648] HybridClock initialized: now 1781837263054279 us; error 27 us; skew 500 ppm
Jun 19 02:47:43 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837263, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:43.056727 2259 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:47:43.057211 2259 webserver.cc:460] Webserver started at http://127.19.113.65:44685/ using document root <none> and password file <none>
I20260619 02:47:43.057456 2259 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:43.057507 2259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:43.058753 2259 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:43.059419 2303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.059572 2259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.059645 2259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "97102225dc5f49b599d442616e438c2e"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "7ee1631d2f82a77cfe56fd64f66f8c1b"
server_key_iv: "d7888b74df2fc31411281941d2911977"
server_key_version: "encryptionkey@0"
I20260619 02:47:43.059993 2259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:43.071718 2259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:43.074137 2259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:43.074290 2259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:43.074563 2259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:43.075106 2310 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:47:43.076496 2259 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:47:43.076555 2259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:47:43.076587 2259 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:47:43.077344 2259 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:47:43.077378 2259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:47:43.077450 2310 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Bootstrap starting.
I20260619 02:47:43.083822 2259 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:39997
I20260619 02:47:43.084174 2259 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:47:43.084604 2417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:39997 every 8 connection(s)
I20260619 02:47:43.087131 2310 log.cc:824] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:43.088032 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 2259
I20260619 02:47:43.088261 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:42321
--local_ip_for_outbound_sockets=127.19.113.66
--tserver_master_addrs=127.19.113.126:46823
--webserver_port=36687
--webserver_interface=127.19.113.66
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:47:43.090106 2310 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:47:43.090376 2310 tablet_bootstrap.cc:492] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Bootstrap complete.
I20260619 02:47:43.091442 2310 ts_tablet_manager.cc:1397] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.010s
I20260619 02:47:43.092886 2310 raft_consensus.cc:348] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:43.093206 2310 raft_consensus.cc:729] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97102225dc5f49b599d442616e438c2e, State: Initialized, Role: FOLLOWER
I20260619 02:47:43.093331 2310 consensus_queue.cc:260] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:43.093575 2310 ts_tablet_manager.cc:1428] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260619 02:47:43.174057 2422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:43.174250 2422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:43.174280 2422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:43.175418 2422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:43.175457 2422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:43.175493 2422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:47:43.176872 2422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:42321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=36687
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:43.177088 2422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:43.177284 2422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:43.182133 2428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:43.182293 2429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:43.182142 2431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:43.182564 2422 server_base.cc:1034] running on GCE node
I20260619 02:47:43.182770 2422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:43.183043 2422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:43.184204 2422 hybrid_clock.cc:648] HybridClock initialized: now 1781837263184188 us; error 31 us; skew 500 ppm
W20260619 02:47:43.185215 2420 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:47:43.084796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:50439)
0619 02:47:43.084960 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.085001 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:43.085420 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:43.185012 (+ 99592us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
W20260619 02:47:43.185473 2418 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:46823 (0 consecutive failures): Network error: Failed to ping master at 127.19.113.126:46823: Client connection negotiation failed: client connection to 127.19.113.126:46823: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:47:43.185751 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:43.185652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:48981)
0619 02:47:43.185696 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.185732 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 02:47:43.186064 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:43.185987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:58261)
0619 02:47:43.186026 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.186047 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 02:47:43.186177 2418 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 02:47:43 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837263, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:43.186964 2422 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:47:43.187357 2422 webserver.cc:460] Webserver started at http://127.19.113.66:36687/ using document root <none> and password file <none>
I20260619 02:47:43.187541 2422 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:43.187583 2422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:43.188849 2422 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:43.189370 2438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.189502 2422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:43.189571 2422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "635262b47d2940198774651e5d93abcf"
format_stamp: "Formatted at 2026-06-19 02:47:41 on dist-test-slave-zqpb"
server_key: "678ac7be66d0c4187e0e8e256ea55746"
server_key_iv: "88013af157c6f53434e239159b27fd19"
server_key_version: "encryptionkey@0"
I20260619 02:47:43.189940 2422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:43.210781 2422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:43.212675 2422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:43.212786 2422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:43.213020 2422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:43.213306 2422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:43.213346 2422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.213376 2422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:43.213399 2422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.218904 2422 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:42321
I20260619 02:47:43.218982 2551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:42321 every 8 connection(s)
I20260619 02:47:43.219271 2422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:47:43.220238 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:43.219885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:57625)
0619 02:47:43.220028 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.220107 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
W20260619 02:47:43.220583 2552 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:46823 (0 consecutive failures): Network error: Failed to ping master at 127.19.113.126:46823: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
I20260619 02:47:43.220808 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:43.220743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:44579)
0619 02:47:43.220769 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.220793 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260619 02:47:43.221037 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:43.220992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:57301)
0619 02:47:43.221011 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.221027 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260619 02:47:43.221110 2552 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:47:43.227221 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 2422
I20260619 02:47:43.227429 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:33609
--local_ip_for_outbound_sockets=127.19.113.67
--tserver_master_addrs=127.19.113.126:46823
--webserver_port=33777
--webserver_interface=127.19.113.67
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:43.306406 2555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:43.306598 2555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:43.306628 2555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:43.307786 2555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:43.307826 2555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:43.307861 2555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:47:43.309291 2555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:33609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=33777
--tserver_master_addrs=127.19.113.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:43.309530 2555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:43.309731 2555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:43.314772 2560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:43.314682 2561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:43.314692 2555 server_base.cc:1034] running on GCE node
W20260619 02:47:43.314700 2563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:43.315132 2555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:43.315354 2555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:43.316561 2555 hybrid_clock.cc:648] HybridClock initialized: now 1781837263316544 us; error 26 us; skew 500 ppm
Jun 19 02:47:43 dist-test-slave-zqpb krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837263, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:43.318936 2555 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:47:43.319336 2555 webserver.cc:460] Webserver started at http://127.19.113.67:33777/ using document root <none> and password file <none>
I20260619 02:47:43.319545 2555 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:43.319593 2555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:43.320811 2555 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:47:43.321317 2570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.321482 2555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.321563 2555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "344aa666d8374552810e4a83c80d5e38"
format_stamp: "Formatted at 2026-06-19 02:47:42 on dist-test-slave-zqpb"
server_key: "42567174a32fc86644dea421a039cac8"
server_key_iv: "442e8ae62ec257db258feaf2f9eb1df5"
server_key_version: "encryptionkey@0"
I20260619 02:47:43.321866 2555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:43.348527 2555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:43.350309 2555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:43.350440 2555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:43.350628 2555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:43.350903 2555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:43.350932 2555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.350953 2555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:43.350965 2555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:43.356354 2555 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:33609
I20260619 02:47:43.356426 2683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:33609 every 8 connection(s)
I20260619 02:47:43.356736 2555 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:47:43.357725 2686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:43.357365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:57675)
0619 02:47:43.357514 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.357591 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
W20260619 02:47:43.358104 2684 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:46823 (0 consecutive failures): Network error: Failed to ping master at 127.19.113.126:46823: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
I20260619 02:47:43.358361 2686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:43.358272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:48647)
0619 02:47:43.358317 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.358343 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:47:43.358620 2686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:43.358555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:51859)
0619 02:47:43.358588 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:43.358607 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 02:47:43.358713 2684 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:47:43.361966 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 2555
I20260619 02:47:44.186782 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:44.186554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:44775)
0619 02:47:44.186678 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:44.186717 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260619 02:47:44.221634 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:44.221420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:52159)
0619 02:47:44.221545 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:44.221579 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260619 02:47:44.359189 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:44.358987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:37383)
0619 02:47:44.359095 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:44.359134 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260619 02:47:44.601796 2691 raft_consensus.cc:482] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:44.601919 2691 raft_consensus.cc:504] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:44.602238 2691 leader_election.cc:288] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609), 635262b47d2940198774651e5d93abcf (127.19.113.66:42321)
Jun 19 02:47:44 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837263, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.66@KRBTEST.COM
Jun 19 02:47:44 dist-test-slave-zqpb krb5kdc[1123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837263, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.67@KRBTEST.COM
I20260619 02:47:44.612356 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:44.602376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:33609 (local address 127.19.113.65:53057)
0619 02:47:44.602423 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:44.602435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:44.602509 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:44.603428 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:44.603431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:44.603792 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:44.603987 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:44.603993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:44.605064 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:44.605066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:44.605871 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:44.605873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:44.606432 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:44.606433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:44.606547 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:44.607098 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:44.607106 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:44.608991 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:44.610991 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:44.610992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:44.610992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:44.611170 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:44.611434 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:44.611435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:44.611435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:44.611501 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:44.611909 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:44.611910 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:44.611981 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 02:47:44.612068 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:47:44.612261 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":13}
I20260619 02:47:44.612366 2694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:44.602604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:49011 (local address 127.19.113.66:42321)
0619 02:47:44.602756 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:44.602765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:44.602791 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:47:44.603227 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:44.603229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:44.603266 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:44.603398 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:44.604099 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:44.604978 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:44.605961 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:44.606360 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:44.606455 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:44.608749 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:44.608754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:44.608754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:44.608767 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:44.610638 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:44.611130 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:44.611134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:44.611134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:44.611247 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:44.611516 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:44.611517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:44.611517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:44.611775 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:44.611837 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:44.612133 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 02:47:44.612251 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":53,"threads_started":1}
I20260619 02:47:44.612370 2693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:44.602526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:53057 (local address 127.19.113.67:33609)
0619 02:47:44.602688 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:44.602698 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:44.602730 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:47:44.603209 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:44.603212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:44.603255 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:44.603383 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:44.604113 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:44.604948 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:44.605956 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:44.606352 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:44.606454 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:44.609086 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:44.609090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:44.609091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:44.609100 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:44.610861 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:44.611249 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:44.611252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:44.611252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:44.611369 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:44.611559 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:44.611559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:44.611560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:44.611790 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:44.611928 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:44.612135 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 02:47:44.612255 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":64,"threads_started":1}
I20260619 02:47:44.612478 2692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:44.602481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:42321 (local address 127.19.113.65:49011)
0619 02:47:44.602604 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:44.602621 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:44.602675 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:44.603428 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:44.603431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:44.603791 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:44.603987 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:44.603993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:44.605079 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:44.605080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:44.605871 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:44.605874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:44.606435 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:44.606436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:44.606545 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:44.607129 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:44.607133 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:44.608637 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:44.610726 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:44.610734 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:44.610736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:44.611051 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:44.611311 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:44.611312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:44.611312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:44.611459 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:44.611839 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:44.611845 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:44.611982 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 02:47:44.612068 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:47:44.612353 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":200,"thread_start_us":63,"threads_started":1}
W20260619 02:47:44.613790 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
W20260619 02:47:44.613865 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 635262b47d2940198774651e5d93abcf (127.19.113.66:42321): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
I20260619 02:47:44.613890 2304 leader_election.cc:302] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97102225dc5f49b599d442616e438c2e; no voters: 344aa666d8374552810e4a83c80d5e38, 635262b47d2940198774651e5d93abcf
I20260619 02:47:44.613981 2691 raft_consensus.cc:2726] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:47:45.187486 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:45.187270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:39705)
0619 02:47:45.187400 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:45.187440 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260619 02:47:45.222226 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:45.222037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:53661)
0619 02:47:45.222154 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:45.222191 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260619 02:47:45.359840 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:45.359614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:52213)
0619 02:47:45.359754 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:45.359796 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260619 02:47:46.188158 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:46.187931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:45011)
0619 02:47:46.188068 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:46.188111 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260619 02:47:46.222766 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:46.222576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:41453)
0619 02:47:46.222694 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:46.222727 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260619 02:47:46.360491 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:46.360308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:59935)
0619 02:47:46.360416 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:46.360450 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260619 02:47:46.399818 2701 raft_consensus.cc:482] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:46.399911 2701 raft_consensus.cc:504] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:46.400035 2701 leader_election.cc:288] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609), 635262b47d2940198774651e5d93abcf (127.19.113.66:42321)
W20260619 02:47:46.400450 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
W20260619 02:47:46.400534 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 635262b47d2940198774651e5d93abcf (127.19.113.66:42321): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
I20260619 02:47:46.400606 2304 leader_election.cc:302] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97102225dc5f49b599d442616e438c2e; no voters: 344aa666d8374552810e4a83c80d5e38, 635262b47d2940198774651e5d93abcf
I20260619 02:47:46.400707 2701 raft_consensus.cc:2726] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:47:47.188899 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:47.188608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:60271)
0619 02:47:47.188752 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:47.188840 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
I20260619 02:47:47.223376 2703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:47.223167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:46939)
0619 02:47:47.223297 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:47.223336 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260619 02:47:47.361122 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:47.360936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:55493)
0619 02:47:47.361043 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:47.361078 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260619 02:47:48.189667 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:48.189432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:42101)
0619 02:47:48.189554 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:48.189609 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260619 02:47:48.223971 2706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:48.223773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:58611)
0619 02:47:48.223893 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:48.223930 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260619 02:47:48.361869 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:48.361600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:45637)
0619 02:47:48.361773 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:48.361818 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":84,"threads_started":1}
I20260619 02:47:49.190311 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:49.190078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:50421)
0619 02:47:49.190227 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:49.190266 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260619 02:47:49.224517 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:49.224332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:35379)
0619 02:47:49.224443 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:49.224477 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260619 02:47:49.362519 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:49.362322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:50455)
0619 02:47:49.362439 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:49.362473 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260619 02:47:49.447017 2711 raft_consensus.cc:482] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:49.447111 2711 raft_consensus.cc:504] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:49.447263 2711 leader_election.cc:288] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609), 635262b47d2940198774651e5d93abcf (127.19.113.66:42321)
W20260619 02:47:49.447702 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 635262b47d2940198774651e5d93abcf (127.19.113.66:42321): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
W20260619 02:47:49.447778 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
I20260619 02:47:49.447803 2304 leader_election.cc:302] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97102225dc5f49b599d442616e438c2e; no voters: 344aa666d8374552810e4a83c80d5e38, 635262b47d2940198774651e5d93abcf
I20260619 02:47:49.447890 2711 raft_consensus.cc:2726] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:47:50.190943 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:50.190713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:52779)
0619 02:47:50.190829 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:50.190889 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260619 02:47:50.225119 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:50.224917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:42063)
0619 02:47:50.225038 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:50.225075 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260619 02:47:50.363175 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:50.362972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:46671)
0619 02:47:50.363085 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:50.363125 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260619 02:47:51.191565 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:51.191355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:47707)
0619 02:47:51.191483 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:51.191517 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260619 02:47:51.225710 2716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:51.225507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:37683)
0619 02:47:51.225627 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:51.225664 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260619 02:47:51.363822 2717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:51.363593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:45569)
0619 02:47:51.363733 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:51.363778 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260619 02:47:52.192312 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:52.192050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:35877)
0619 02:47:52.192194 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:52.192263 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260619 02:47:52.226301 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:52.226082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:39343)
0619 02:47:52.226219 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:52.226256 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260619 02:47:52.332010 2720 raft_consensus.cc:482] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:52.332099 2720 raft_consensus.cc:504] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:52.332222 2720 leader_election.cc:288] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609), 635262b47d2940198774651e5d93abcf (127.19.113.66:42321)
W20260619 02:47:52.332607 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 635262b47d2940198774651e5d93abcf (127.19.113.66:42321): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
W20260619 02:47:52.332682 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
I20260619 02:47:52.332703 2304 leader_election.cc:302] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97102225dc5f49b599d442616e438c2e; no voters: 344aa666d8374552810e4a83c80d5e38, 635262b47d2940198774651e5d93abcf
I20260619 02:47:52.332787 2720 raft_consensus.cc:2726] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:47:52.364472 2721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:52.364264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:55963)
0619 02:47:52.364391 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:52.364429 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260619 02:47:53.192992 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:53.192767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:58689)
0619 02:47:53.192902 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:53.192945 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260619 02:47:53.227003 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:53.226743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:40069)
0619 02:47:53.226915 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:53.226958 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":67,"threads_started":1}
I20260619 02:47:53.365149 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:53.364915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:40917)
0619 02:47:53.365055 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:53.365101 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260619 02:47:54.193740 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:54.193482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:58843)
0619 02:47:54.193641 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:54.193691 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":61,"threads_started":1}
I20260619 02:47:54.227695 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:54.227483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:37317)
0619 02:47:54.227609 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:54.227649 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260619 02:47:54.365756 2727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:54.365548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:39137)
0619 02:47:54.365661 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:54.365696 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260619 02:47:55.194448 2728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:55.194224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:43813)
0619 02:47:55.194353 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:55.194401 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260619 02:47:55.228410 2729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:55.228177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:49057)
0619 02:47:55.228330 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:55.228365 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260619 02:47:55.242940 2730 raft_consensus.cc:482] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:47:55.243026 2730 raft_consensus.cc:504] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344aa666d8374552810e4a83c80d5e38" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33609 } } peers { permanent_uuid: "635262b47d2940198774651e5d93abcf" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 42321 } } peers { permanent_uuid: "97102225dc5f49b599d442616e438c2e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 39997 } }
I20260619 02:47:55.243175 2730 leader_election.cc:288] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609), 635262b47d2940198774651e5d93abcf (127.19.113.66:42321)
W20260619 02:47:55.243533 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 635262b47d2940198774651e5d93abcf (127.19.113.66:42321): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
W20260619 02:47:55.243592 2304 leader_election.cc:341] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 344aa666d8374552810e4a83c80d5e38 (127.19.113.67:33609): Not found: Tablet not found: 6f57bcdc2d0c40b5a249ee45350d5b0c
I20260619 02:47:55.243604 2304 leader_election.cc:302] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97102225dc5f49b599d442616e438c2e; no voters: 344aa666d8374552810e4a83c80d5e38, 635262b47d2940198774651e5d93abcf
I20260619 02:47:55.243669 2730 raft_consensus.cc:2726] T 6f57bcdc2d0c40b5a249ee45350d5b0c P 97102225dc5f49b599d442616e438c2e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:47:55.366482 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:55.366223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:52829)
0619 02:47:55.366396 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:55.366439 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260619 02:47:56.195128 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:56.194909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:45197)
0619 02:47:56.195040 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:56.195084 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260619 02:47:56.229144 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:56.228929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:51277)
0619 02:47:56.229056 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:56.229099 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260619 02:47:56.367144 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:56.366946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:53639)
0619 02:47:56.367049 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:56.367092 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":55,"threads_started":1}
I20260619 02:47:57.195822 2735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:57.195591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:53043)
0619 02:47:57.195728 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:57.195771 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260619 02:47:57.229897 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:57.229626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:35325)
0619 02:47:57.229813 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:57.229851 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
I20260619 02:47:57.367856 2737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:57.367608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:58031)
0619 02:47:57.367761 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:57.367808 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260619 02:47:58.196597 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:58.196373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.65:48495)
0619 02:47:58.196508 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:58.196549 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260619 02:47:58.230614 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:58.230398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.66:42099)
0619 02:47:58.230523 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:58.230563 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260619 02:47:58.368592 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:58.368367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:46823 (local address 127.19.113.67:56017)
0619 02:47:58.368500 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:58.368543 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:46823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260619 02:47:58.370699 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 2259
I20260619 02:47:58.374854 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 2422
I20260619 02:47:58.378659 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 2555
2026-06-19T02:47:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (17939 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:47:58 dist-test-slave-zqpb krb5kdc[2743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:47:58 dist-test-slave-zqpb krb5kdc[2743](info): set up 2 sockets
Jun 19 02:47:58 dist-test-slave-zqpb krb5kdc[2743](info): commencing operation
krb5kdc: starting...
W20260619 02:47:59.322810 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:47:59 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:47:59.439580 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45015
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:45015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:59.519402 2756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:59.519609 2756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:59.519631 2756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:59.520812 2756 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:47:59.520855 2756 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:47:59.520869 2756 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:47:59.520888 2756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:47:59.520900 2756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:59.520912 2756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:59.520941 2756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:47:59.520962 2756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:47:59.522501 2756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:45015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:59.522742 2756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:59.522974 2756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:59.527911 2764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:59.527921 2761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:59.527944 2756 server_base.cc:1034] running on GCE node
W20260619 02:47:59.527911 2762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:59.528281 2756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:59.528529 2756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:59.529662 2756 hybrid_clock.cc:648] HybridClock initialized: now 1781837279529645 us; error 31 us; skew 500 ppm
Jun 19 02:47:59 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:59.536541 2756 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:47:59.536938 2756 webserver.cc:460] Webserver started at http://127.19.113.126:32789/ using document root <none> and password file <none>
I20260619 02:47:59.537139 2756 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:59.537179 2756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:59.537271 2756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:59.538278 2756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "0e3cfbd1f02540ecabc9e0abc826149c"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "f5dae2ea45f90b6e5b27fc0d66187e50"
server_key_iv: "a4f9716b69d70587494ef114e6485f4b"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.538573 2756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "0e3cfbd1f02540ecabc9e0abc826149c"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "f5dae2ea45f90b6e5b27fc0d66187e50"
server_key_iv: "a4f9716b69d70587494ef114e6485f4b"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.539829 2756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:47:59.540488 2771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.540632 2756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:47:59.540697 2756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "0e3cfbd1f02540ecabc9e0abc826149c"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "f5dae2ea45f90b6e5b27fc0d66187e50"
server_key_iv: "a4f9716b69d70587494ef114e6485f4b"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.540767 2756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:59.551561 2756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:59.553589 2756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:59.553707 2756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:59.557458 2756 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:45015
I20260619 02:47:59.557497 2823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:45015 every 8 connection(s)
I20260619 02:47:59.557804 2756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:47:59.558611 2824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:47:59.560909 2824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Bootstrap starting.
I20260619 02:47:59.561445 2824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Neither blocks nor log segments found. Creating new log.
I20260619 02:47:59.561702 2824 log.cc:824] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:47:59.562347 2824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: No bootstrap required, opened a new log
I20260619 02:47:59.563979 2824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:47:59.564159 2824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:47:59.564154 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 2756
I20260619 02:47:59.564193 2824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3cfbd1f02540ecabc9e0abc826149c, State: Initialized, Role: FOLLOWER
I20260619 02:47:59.564260 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:47:59.564338 2824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:47:59.564414 2824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:47:59.564443 19909 external_mini_cluster.cc:1428] Setting key dff0c8c06fd32144710dd6274c32547a
I20260619 02:47:59.564448 2824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:47:59.564492 2824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:47:59.565119 2824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:47:59.565207 2824 leader_election.cc:302] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e3cfbd1f02540ecabc9e0abc826149c; no voters:
I20260619 02:47:59.565368 2824 leader_election.cc:288] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:47:59.565407 2829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:47:59.565505 2829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 1 LEADER]: Becoming Leader. State: Replica: 0e3cfbd1f02540ecabc9e0abc826149c, State: Running, Role: LEADER
I20260619 02:47:59.565616 2824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:47:59.565644 2829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:47:59.566227 2830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } } }
I20260619 02:47:59.566304 2830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: This master's current role is: LEADER
I20260619 02:47:59.566918 2831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e3cfbd1f02540ecabc9e0abc826149c. Latest consensus state: current_term: 1 leader_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } } }
I20260619 02:47:59.567160 2831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: This master's current role is: LEADER
W20260619 02:47:59.567345 2845 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:47:59.567416 2845 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:47:59.567463 2838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:47:59.567605 2838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:47:59.568931 2838 catalog_manager.cc:1347] Generated new cluster ID: edceb9c6a9f243fea3edea4768bfe8b7
I20260619 02:47:59.568979 2838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:47:59 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:59.575476 2838 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:47:59.575954 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:59.564906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.19.113.126:45015)
0619 02:47:59.565371 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:59.565387 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:59.565418 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:47:59.565541 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:59.565548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:59.565604 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:59.565741 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:59.566476 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.567847 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.568900 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.569521 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.569625 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:59.572366 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:59.572370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:59.572371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:59.572383 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:59.574269 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:59.574698 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:59.574702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:59.574702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:59.574813 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:59.575020 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:59.575021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:59.575022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:59.575282 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:59.575446 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:59.575575 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 02:47:59.575732 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":388,"threads_started":1}
I20260619 02:47:59.576148 2838 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:47:59.581599 2838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Generated new TSK 0
I20260619 02:47:59.581775 2838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:47:59.608740 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45015
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:59.689407 2850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:59.689616 2850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:59.689639 2850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:59.690825 2850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:59.690860 2850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:59.690914 2850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:47:59.692389 2850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:59.692574 2850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:59.692764 2850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:59.697367 2855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:59.697374 2856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:59.697504 2850 server_base.cc:1034] running on GCE node
W20260619 02:47:59.697575 2858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:59.697813 2850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:59.698056 2850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:59.699187 2850 hybrid_clock.cc:648] HybridClock initialized: now 1781837279699173 us; error 25 us; skew 500 ppm
Jun 19 02:47:59 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:59.701442 2850 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:47:59.701820 2850 webserver.cc:460] Webserver started at http://127.19.113.65:43365/ using document root <none> and password file <none>
I20260619 02:47:59.702049 2850 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:59.702096 2850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:59.702215 2850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:59.703259 2850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "dacbd5b5faff48068d2f3abc90508f6f"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "1b5fe70a7aa7207203e76189ada9bd68"
server_key_iv: "b288c3fa2d3bd794d1d28100389cae80"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.703568 2850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "dacbd5b5faff48068d2f3abc90508f6f"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "1b5fe70a7aa7207203e76189ada9bd68"
server_key_iv: "b288c3fa2d3bd794d1d28100389cae80"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.704720 2850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:47:59.705358 2865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.705497 2850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:47:59.705574 2850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "dacbd5b5faff48068d2f3abc90508f6f"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "1b5fe70a7aa7207203e76189ada9bd68"
server_key_iv: "b288c3fa2d3bd794d1d28100389cae80"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.705629 2850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:59.718250 2850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:59.719898 2850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:59.720021 2850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:59.720211 2850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:59.720477 2850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:59.720510 2850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.720535 2850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:59.720559 2850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.726029 2850 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:36223
I20260619 02:47:59.726107 2978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:36223 every 8 connection(s)
I20260619 02:47:59.726353 2850 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:47:59 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:47:59.733352 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 2850
I20260619 02:47:59.733469 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:47:59.733671 19909 external_mini_cluster.cc:1428] Setting key 3175cd20508d0a5829cd4ba387839742
I20260619 02:47:59.735790 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:59.727007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:46727 (local address 127.19.113.126:45015)
0619 02:47:59.727060 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:59.727062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:59.727233 (+ 171us) server_negotiation.cc:407] Connection header received
0619 02:47:59.727725 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:59.727726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:59.727743 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:59.727791 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:59.728706 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.729213 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.730174 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.730399 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.730454 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:59.732561 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:59.732562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:59.732563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:59.732574 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:59.733903 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:59.734250 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:59.734251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:59.734251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:59.734314 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:59.735133 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:59.735133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:59.735134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:59.735252 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:59.735310 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:59.735643 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 02:47:59.735726 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:47:59.736182 2981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:59.726984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:46727)
0619 02:47:59.727125 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:59.727162 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:59.727633 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:59.727859 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:59.727863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:59.728409 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:59.728596 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:59.728600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.729290 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.729291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:59.730089 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:59.730092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.730469 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.730470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:59.730594 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:59.731089 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:59.731097 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:59.732462 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:59.733971 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:59.733978 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:59.733980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:59.734186 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:59.734934 (+ 748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:59.734935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:59.734936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:59.735066 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:59.735316 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:59.735320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:59.735474 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 02:47:59.735875 (+ 401us) client_negotiation.cc:240] Negotiation successful
0619 02:47:59.736023 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260619 02:47:59.736601 2979 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45015
I20260619 02:47:59.736691 2979 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:59.736860 2979 heartbeater.cc:511] Master 127.19.113.126:45015 requested a full tablet report, sending...
I20260619 02:47:59.737355 2788 ts_manager.cc:194] Registered new tserver with Master: dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223)
I20260619 02:47:59.738260 2788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:46727
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:47:59.757208 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45015
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:59.834951 2984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:59.835156 2984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:59.835180 2984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:59.836390 2984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:59.836429 2984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:59.836454 2984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:47:59.837855 2984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:59.838032 2984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:59.838215 2984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:59.842914 2990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:59.842991 2989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:59.842957 2984 server_base.cc:1034] running on GCE node
W20260619 02:47:59.842927 2992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:59.843309 2984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:59.843545 2984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:59.844693 2984 hybrid_clock.cc:648] HybridClock initialized: now 1781837279844664 us; error 43 us; skew 500 ppm
Jun 19 02:47:59 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:59.846990 2984 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:47:59.847396 2984 webserver.cc:460] Webserver started at http://127.19.113.66:35447/ using document root <none> and password file <none>
I20260619 02:47:59.847589 2984 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:59.847632 2984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:59.847762 2984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:59.848743 2984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "4bc1b6e47d9a4307950ee2a69557ef68"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "e6a4aaa33001032d22b9cb2807f63dc4"
server_key_iv: "634fb0d3981c0f821ab47a9d66818987"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.849067 2984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "4bc1b6e47d9a4307950ee2a69557ef68"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "e6a4aaa33001032d22b9cb2807f63dc4"
server_key_iv: "634fb0d3981c0f821ab47a9d66818987"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.850260 2984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:47:59.850968 2999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.851120 2984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.851202 2984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "4bc1b6e47d9a4307950ee2a69557ef68"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "e6a4aaa33001032d22b9cb2807f63dc4"
server_key_iv: "634fb0d3981c0f821ab47a9d66818987"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.851267 2984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:59.862078 2984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:59.863906 2984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:59.864012 2984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:59.864192 2984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:59.864437 2984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:59.864465 2984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.864485 2984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:59.864500 2984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.869755 2984 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:45427
I20260619 02:47:59.869793 3112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:45427 every 8 connection(s)
I20260619 02:47:59.870103 2984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:47:59.871384 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 2984
I20260619 02:47:59.871476 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:47:59.871647 19909 external_mini_cluster.cc:1428] Setting key cc8e80891a2b29070893e1022ddc17ee
Jun 19 02:47:59 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
I20260619 02:47:59.879558 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:59.870743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:55685 (local address 127.19.113.126:45015)
0619 02:47:59.870802 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:47:59.870804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:47:59.871005 (+ 201us) server_negotiation.cc:407] Connection header received
0619 02:47:59.871503 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:47:59.871504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:47:59.871518 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:47:59.871569 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:47:59.872478 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.872962 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.874310 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.874505 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.874561 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:59.876835 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:47:59.876836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:47:59.876836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:47:59.876843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:47:59.878174 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:59.878596 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:59.878596 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:59.878597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:59.878640 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:47:59.878928 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:47:59.878929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:47:59.878929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:47:59.879026 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:47:59.879072 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:47:59.879368 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 02:47:59.879443 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:47:59.879738 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:47:59.870704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:55685)
0619 02:47:59.870887 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:47:59.870929 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:47:59.871407 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:47:59.871649 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:47:59.871653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:47:59.872198 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:47:59.872383 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:59.872388 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.873040 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.873041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:59.874224 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:47:59.874227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:47:59.874577 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:47:59.874578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:47:59.874708 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:47:59.875183 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:47:59.875192 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:47:59.876752 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:47:59.878251 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:59.878259 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:59.878262 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:59.878527 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:59.878692 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:47:59.878692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:47:59.878693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:47:59.878828 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:47:59.879086 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:47:59.879092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:47:59.879244 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 02:47:59.879346 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:47:59.879512 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260619 02:47:59.880141 3113 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45015
I20260619 02:47:59.880230 3113 heartbeater.cc:464] Registering TS with master...
I20260619 02:47:59.880435 3113 heartbeater.cc:511] Master 127.19.113.126:45015 requested a full tablet report, sending...
I20260619 02:47:59.880753 2788 ts_manager.cc:194] Registered new tserver with Master: 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427)
I20260619 02:47:59.881083 2788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:55685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:47:59.894943 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45015
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:47:59.972507 3118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:47:59.972697 3118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:47:59.972718 3118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:47:59.973794 3118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:47:59.973824 3118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:47:59.973848 3118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:47:59.975234 3118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:47:59.975404 3118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:47:59.975575 3118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:47:59.980105 3123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:59.980253 3126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:47:59.980237 3124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:47:59.980422 3118 server_base.cc:1034] running on GCE node
I20260619 02:47:59.980603 3118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:47:59.980824 3118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:47:59.981958 3118 hybrid_clock.cc:648] HybridClock initialized: now 1781837279981943 us; error 29 us; skew 500 ppm
Jun 19 02:47:59 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:47:59.984373 3118 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:47:59.984758 3118 webserver.cc:460] Webserver started at http://127.19.113.67:39185/ using document root <none> and password file <none>
I20260619 02:47:59.984959 3118 fs_manager.cc:359] Metadata directory not provided
I20260619 02:47:59.985005 3118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:47:59.985127 3118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:47:59.986172 3118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "3e828485ea58447f92a413721d0e91d5"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "abc6bb6670ec1f4c53929e35cb46866c"
server_key_iv: "8577581cf53fb48e1d562bdf675a84ef"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.986490 3118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "3e828485ea58447f92a413721d0e91d5"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "abc6bb6670ec1f4c53929e35cb46866c"
server_key_iv: "8577581cf53fb48e1d562bdf675a84ef"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.987767 3118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:47:59.988425 3133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.988557 3118 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:47:59.988626 3118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "3e828485ea58447f92a413721d0e91d5"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "abc6bb6670ec1f4c53929e35cb46866c"
server_key_iv: "8577581cf53fb48e1d562bdf675a84ef"
server_key_version: "encryptionkey@0"
I20260619 02:47:59.988698 3118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:47:59.996660 3118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:47:59.998287 3118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:47:59.998406 3118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:47:59.998610 3118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:47:59.998899 3118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:47:59.998940 3118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:47:59.998973 3118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:47:59.998997 3118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:00.004563 3118 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:45509
I20260619 02:48:00.004599 3246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:45509 every 8 connection(s)
I20260619 02:48:00.004884 3118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:48:00.009128 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 3118
I20260619 02:48:00.009232 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:48:00.009424 19909 external_mini_cluster.cc:1428] Setting key 81ec914c5ac6356679b8b41fe16cac46
Jun 19 02:48:00 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:48:00.013893 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.005516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:47007 (local address 127.19.113.126:45015)
0619 02:48:00.005558 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:00.005559 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:00.005755 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:48:00.006188 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:00.006188 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:00.006198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:00.006249 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:00.007126 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.007623 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.008779 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.009019 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.009094 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.011322 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:00.011323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:00.011324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:00.011331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:00.012634 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:00.013028 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:00.013029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:00.013029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:00.013083 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:00.013301 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:00.013302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:00.013303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:00.013407 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:00.013464 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:00.013714 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 02:48:00.013801 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:48:00.014060 3249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.005492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:47007)
0619 02:48:00.005633 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:00.005666 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:00.006115 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:00.006299 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:00.006302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:00.006841 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:00.007045 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.007050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.007750 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.007751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.008675 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.008678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.009095 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.009096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.009208 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.009737 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:00.009747 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:00.011234 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:00.012746 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:00.012753 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:00.012755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:00.012963 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:00.013137 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:00.013137 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:00.013138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:00.013247 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:00.013461 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:00.013468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:00.013600 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:48:00.013686 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:48:00.013842 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260619 02:48:00.014520 3247 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45015
I20260619 02:48:00.014601 3247 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:00.014761 3247 heartbeater.cc:511] Master 127.19.113.126:45015 requested a full tablet report, sending...
I20260619 02:48:00.015141 2788 ts_manager.cc:194] Registered new tserver with Master: 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509)
I20260619 02:48:00.015501 2788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:47007
I20260619 02:48:00.021090 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:48:00.027730 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.022367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.19.113.126:45015)
0619 02:48:00.022420 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:00.022421 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:00.022514 (+ 93us) server_negotiation.cc:407] Connection header received
0619 02:48:00.022555 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:00.022556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:00.022564 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:00.022607 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:00.023197 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.023680 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.024376 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.024555 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.024604 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.025634 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:00.025636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:00.025636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:00.025641 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:00.026885 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:00.027179 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:00.027179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:00.027180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:00.027226 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:00.027347 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:00.027348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:00.027348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:00.027474 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:00.027541 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:00.027611 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:48:00.027667 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:48:00.029129 2788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:48:00.030616 2788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:48:00.036664 3259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.032715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:36223 (local address 127.0.0.1:52604)
0619 02:48:00.032985 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:00.032994 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:00.033030 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:00.033571 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:00.033571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:00.033573 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:00.033608 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.033609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.034707 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.034708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.035594 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.035596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.036450 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.036451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.036517 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.036527 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 02:48:00.036574 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:48:00.036610 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":98,"threads_started":1}
I20260619 02:48:00.036725 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.032947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.19.113.65:36223)
0619 02:48:00.033407 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:00.033411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:00.033420 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:48:00.033456 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:00.033457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:00.033473 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:00.033567 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:00.033661 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.034625 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.035677 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.036371 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.036439 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.036495 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:00.036605 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:48:00.036648 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":81,"threads_started":1}
I20260619 02:48:00.037037 3256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.032535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:45509 (local address 127.0.0.1:42532)
0619 02:48:00.032758 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:00.032802 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:00.032867 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:00.033296 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:00.033297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:00.033298 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:00.033333 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.033335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.034259 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.034260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.035562 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.035564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.036840 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.036841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.036895 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.036899 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:48:00.036937 (+ 38us) client_negotiation.cc:240] Negotiation successful
0619 02:48:00.036971 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":3,"thread_start_us":52,"threads_started":1}
I20260619 02:48:00.037277 3257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.032648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.19.113.67:45509)
0619 02:48:00.033137 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:00.033141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:00.033149 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:48:00.033180 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:00.033182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:00.033196 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:00.033282 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:00.033407 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.034175 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.035642 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.036371 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.036441 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.036495 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:00.037161 (+ 666us) server_negotiation.cc:299] Negotiation successful
0619 02:48:00.037202 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":80,"threads_started":1}
I20260619 02:48:00.037349 2913 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1d6444ac804d7ba97a360016858f0a (DEFAULT_TABLE table=test-table [id=a6be668316154850a4dd0dbc8d3c9115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:00.037526 3255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.032549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:45427 (local address 127.0.0.1:35764)
0619 02:48:00.032822 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:00.032832 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:00.032881 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:00.033072 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:00.033074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:00.033077 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:00.033142 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.033144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.034475 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.034476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.035561 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.035563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.037256 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.037257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.037379 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.037383 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:48:00.037434 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:48:00.037468 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":99,"threads_started":1}
I20260619 02:48:00.037571 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.032714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.19.113.66:45427)
0619 02:48:00.032902 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:00.032907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:00.032929 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:48:00.032966 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:00.032968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:00.032984 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:00.033060 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:00.033209 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.034381 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.036530 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.037184 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.037253 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.037302 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:00.037456 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 02:48:00.037499 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260619 02:48:00.037709 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1d6444ac804d7ba97a360016858f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:00.037935 3181 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1d6444ac804d7ba97a360016858f0a (DEFAULT_TABLE table=test-table [id=a6be668316154850a4dd0dbc8d3c9115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:00.038213 3181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1d6444ac804d7ba97a360016858f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:00.040923 3261 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Bootstrap starting.
I20260619 02:48:00.041404 3262 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Bootstrap starting.
I20260619 02:48:00.041565 3261 tablet_bootstrap.cc:654] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:00.041563 3047 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1d6444ac804d7ba97a360016858f0a (DEFAULT_TABLE table=test-table [id=a6be668316154850a4dd0dbc8d3c9115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:00.041890 3047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1d6444ac804d7ba97a360016858f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:00.041985 3262 tablet_bootstrap.cc:654] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:00.042164 3261 log.cc:824] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:00.042497 3262 log.cc:824] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:00.042955 3261 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: No bootstrap required, opened a new log
I20260619 02:48:00.043022 3261 ts_tablet_manager.cc:1397] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:48:00.043246 3262 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: No bootstrap required, opened a new log
I20260619 02:48:00.043304 3262 ts_tablet_manager.cc:1397] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:48:00.044703 3262 raft_consensus.cc:348] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.044839 3262 raft_consensus.cc:374] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:00.044870 3262 raft_consensus.cc:729] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e828485ea58447f92a413721d0e91d5, State: Initialized, Role: FOLLOWER
I20260619 02:48:00.045002 3262 consensus_queue.cc:260] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.044963 3261 raft_consensus.cc:348] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.045113 3261 raft_consensus.cc:374] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:00.045185 3261 raft_consensus.cc:729] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dacbd5b5faff48068d2f3abc90508f6f, State: Initialized, Role: FOLLOWER
I20260619 02:48:00.045286 3262 ts_tablet_manager.cc:1428] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:48:00.045389 3261 consensus_queue.cc:260] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.045502 3247 heartbeater.cc:503] Master 127.19.113.126:45015 was elected leader, sending a full tablet report...
I20260619 02:48:00.045639 3265 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Bootstrap starting.
I20260619 02:48:00.045667 2979 heartbeater.cc:503] Master 127.19.113.126:45015 was elected leader, sending a full tablet report...
I20260619 02:48:00.045866 3261 ts_tablet_manager.cc:1428] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:48:00.046238 3265 tablet_bootstrap.cc:654] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:00.046780 3265 log.cc:824] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:00.047392 3265 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: No bootstrap required, opened a new log
I20260619 02:48:00.047441 3265 ts_tablet_manager.cc:1397] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:48:00.048683 3265 raft_consensus.cc:348] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.048820 3265 raft_consensus.cc:374] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:00.048840 3265 raft_consensus.cc:729] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bc1b6e47d9a4307950ee2a69557ef68, State: Initialized, Role: FOLLOWER
I20260619 02:48:00.048913 3265 consensus_queue.cc:260] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.049085 3265 ts_tablet_manager.cc:1428] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:48:00.049124 3113 heartbeater.cc:503] Master 127.19.113.126:45015 was elected leader, sending a full tablet report...
W20260619 02:48:00.120644 3114 tablet.cc:2369] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:48:00.227037 2980 tablet.cc:2369] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:48:00.255394 3248 tablet.cc:2369] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:48:00.295981 3269 raft_consensus.cc:482] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:48:00.296098 3269 raft_consensus.cc:504] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.296473 3269 leader_election.cc:288] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509), dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223)
I20260619 02:48:00.299247 3257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.296753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:48245 (local address 127.19.113.67:45509)
0619 02:48:00.296830 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:00.296833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:00.296843 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:48:00.296872 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:00.296872 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:00.296881 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:00.296929 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:00.297066 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.297548 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.298516 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.298934 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.298997 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.299007 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:00.299142 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:48:00.299182 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:48:00.299281 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.296666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:45509 (local address 127.19.113.66:48245)
0619 02:48:00.296723 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:00.296736 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:00.296792 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:00.296949 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:00.296950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:00.296952 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:00.297012 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.297013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.297622 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.297623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.298435 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.298437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.299000 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.299001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.299083 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.299086 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:00.299130 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:48:00.299169 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":18}
I20260619 02:48:00.299293 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.296868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:56685 (local address 127.19.113.65:36223)
0619 02:48:00.296920 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:00.296922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:00.297009 (+ 87us) server_negotiation.cc:407] Connection header received
0619 02:48:00.297075 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:00.297076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:00.297088 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:00.297131 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:00.297275 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.297715 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.298580 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.298987 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.299048 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.299055 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:00.299190 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:48:00.299222 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:48:00.299302 3270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.296792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:36223 (local address 127.19.113.66:56685)
0619 02:48:00.296952 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:00.296961 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:00.297011 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:00.297138 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:00.297138 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:00.297139 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:00.297211 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.297212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.297782 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.297783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.298506 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:00.298508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.299054 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.299055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:00.299127 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.299130 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:00.299170 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 02:48:00.299207 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260619 02:48:00.299546 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d6444ac804d7ba97a360016858f0a" candidate_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dacbd5b5faff48068d2f3abc90508f6f" is_pre_election: true
I20260619 02:48:00.299618 3201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d6444ac804d7ba97a360016858f0a" candidate_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e828485ea58447f92a413721d0e91d5" is_pre_election: true
I20260619 02:48:00.299698 2933 raft_consensus.cc:2454] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bc1b6e47d9a4307950ee2a69557ef68 in term 0.
I20260619 02:48:00.299774 3201 raft_consensus.cc:2454] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bc1b6e47d9a4307950ee2a69557ef68 in term 0.
I20260619 02:48:00.299901 3000 leader_election.cc:302] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bc1b6e47d9a4307950ee2a69557ef68, dacbd5b5faff48068d2f3abc90508f6f; no voters:
I20260619 02:48:00.300025 3269 raft_consensus.cc:2781] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:48:00.300096 3269 raft_consensus.cc:482] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:48:00.300112 3269 raft_consensus.cc:3037] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:00.300812 3269 raft_consensus.cc:504] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.300916 3269 leader_election.cc:288] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [CANDIDATE]: Term 1 election: Requested vote from peers 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509), dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223)
I20260619 02:48:00.301128 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d6444ac804d7ba97a360016858f0a" candidate_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dacbd5b5faff48068d2f3abc90508f6f"
I20260619 02:48:00.301126 3201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d6444ac804d7ba97a360016858f0a" candidate_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e828485ea58447f92a413721d0e91d5"
I20260619 02:48:00.301213 2933 raft_consensus.cc:3037] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:00.301213 3201 raft_consensus.cc:3037] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:00.302071 3201 raft_consensus.cc:2454] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bc1b6e47d9a4307950ee2a69557ef68 in term 1.
I20260619 02:48:00.302071 2933 raft_consensus.cc:2454] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bc1b6e47d9a4307950ee2a69557ef68 in term 1.
I20260619 02:48:00.302244 3000 leader_election.cc:302] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bc1b6e47d9a4307950ee2a69557ef68, dacbd5b5faff48068d2f3abc90508f6f; no voters:
I20260619 02:48:00.302346 3269 raft_consensus.cc:2781] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:48:00.302476 3269 raft_consensus.cc:686] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 LEADER]: Becoming Leader. State: Replica: 4bc1b6e47d9a4307950ee2a69557ef68, State: Running, Role: LEADER
I20260619 02:48:00.302559 3269 consensus_queue.cc:237] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:00.303418 2788 catalog_manager.cc:5495] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66). New cstate: current_term: 1 leader_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260619 02:48:00.318955 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.316583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.19.113.66:45427)
0619 02:48:00.316625 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:00.316627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:00.316636 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:48:00.316691 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:00.316692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:00.316708 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:00.316789 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:48:00.316898 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.317350 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.318112 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:00.318331 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:00.318392 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:00.318576 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:48:00.318680 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:48:00.318793 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:00.318815 (+ 22us) server_negotiation.cc:299] Negotiation successful
0619 02:48:00.318835 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:48:00.320660 2933 raft_consensus.cc:1261] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Refusing update from remote peer 4bc1b6e47d9a4307950ee2a69557ef68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:48:00.320660 3201 raft_consensus.cc:1261] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 1 FOLLOWER]: Refusing update from remote peer 4bc1b6e47d9a4307950ee2a69557ef68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:48:00.321043 3271 consensus_queue.cc:1035] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:48:00.321167 3269 consensus_queue.cc:1035] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:48:00.323364 3275 mvcc.cc:205] Tried to move back new op lower bound from 7298405500190859264 to 7298405500119924736. Current Snapshot: MvccSnapshot[applied={T|T < 7298405500190859264}]
I20260619 02:48:00.325667 2787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:48:00.325775 2787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:48:00.326524 2787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Sending DeleteTablet for 3 replicas of tablet 6b1d6444ac804d7ba97a360016858f0a
I20260619 02:48:00.326831 3181 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1d6444ac804d7ba97a360016858f0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:00 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:42532
I20260619 02:48:00.326894 2913 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1d6444ac804d7ba97a360016858f0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:00 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:52604
I20260619 02:48:00.327050 3047 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1d6444ac804d7ba97a360016858f0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:00 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:35764
I20260619 02:48:00.327100 3284 tablet_replica.cc:330] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: stopping tablet replica
I20260619 02:48:00.327208 3284 raft_consensus.cc:2229] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:48:00.327242 3283 tablet_replica.cc:330] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: stopping tablet replica
I20260619 02:48:00.327301 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 2850
I20260619 02:48:00.327333 3284 raft_consensus.cc:2258] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:00.327353 3285 tablet_replica.cc:330] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: stopping tablet replica
I20260619 02:48:00.327425 3285 raft_consensus.cc:2229] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:48:00.327502 3285 raft_consensus.cc:2258] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:00.327623 3283 raft_consensus.cc:2229] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:48:00.327761 3283 raft_consensus.cc:2258] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:00.328279 3285 ts_tablet_manager.cc:1905] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:00.330044 3283 ts_tablet_manager.cc:1905] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:00.330071 3285 ts_tablet_manager.cc:1918] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:00.330129 3285 log.cc:1196] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/6b1d6444ac804d7ba97a360016858f0a
I20260619 02:48:00.330457 3285 ts_tablet_manager.cc:1939] T 6b1d6444ac804d7ba97a360016858f0a P 4bc1b6e47d9a4307950ee2a69557ef68: Deleting consensus metadata
I20260619 02:48:00.330889 2773 catalog_manager.cc:4841] TS 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427): tablet 6b1d6444ac804d7ba97a360016858f0a (table test-table [id=a6be668316154850a4dd0dbc8d3c9115]) successfully deleted
I20260619 02:48:00.331671 3283 ts_tablet_manager.cc:1918] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:00.331753 3283 log.cc:1196] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/6b1d6444ac804d7ba97a360016858f0a
I20260619 02:48:00.332048 3283 ts_tablet_manager.cc:1939] T 6b1d6444ac804d7ba97a360016858f0a P 3e828485ea58447f92a413721d0e91d5: Deleting consensus metadata
I20260619 02:48:00.332464 2772 catalog_manager.cc:4841] TS 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509): tablet 6b1d6444ac804d7ba97a360016858f0a (table test-table [id=a6be668316154850a4dd0dbc8d3c9115]) successfully deleted
W20260619 02:48:00.332890 2772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:36223 (error 108)
I20260619 02:48:00.332993 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 2984
I20260619 02:48:00.335276 3255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:00.335117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:36223 (local address 127.0.0.1:52610)
0619 02:48:00.335193 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:00.335243 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:36223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 02:48:00.335415 2772 catalog_manager.cc:4584] TS dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b1d6444ac804d7ba97a360016858f0a: Network error: Client connection negotiation failed: client connection to 127.19.113.65:36223: connect: Connection refused (error 111)
I20260619 02:48:00.337333 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 3118
I20260619 02:48:00.341425 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 2756
I20260619 02:48:20.345847 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45015
--webserver_interface=127.19.113.126
--webserver_port=32789
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:45015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:48:20.427489 3287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:20.427712 3287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:20.427742 3287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:20.429140 3287 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:48:20.429205 3287 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:48:20.429229 3287 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:48:20.429245 3287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:48:20.429261 3287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:20.429280 3287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:20.429303 3287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:48:20.429322 3287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:48:20.431558 3287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:45015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=32789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:20.431941 3287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:20.432191 3287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:48:20.442490 3287 server_base.cc:1034] running on GCE node
W20260619 02:48:20.442457 3296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:20.442644 3293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:20.442457 3294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:20.443051 3287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:20.443405 3287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:20.444578 3287 hybrid_clock.cc:648] HybridClock initialized: now 1781837300444556 us; error 32 us; skew 500 ppm
Jun 19 02:48:20 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837300, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:20.453030 3287 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:48:20.453585 3287 webserver.cc:460] Webserver started at http://127.19.113.126:32789/ using document root <none> and password file <none>
I20260619 02:48:20.453825 3287 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:20.453886 3287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:20.455335 3287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:48:20.456038 3303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.456200 3287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:48:20.456295 3287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "0e3cfbd1f02540ecabc9e0abc826149c"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "f5dae2ea45f90b6e5b27fc0d66187e50"
server_key_iv: "a4f9716b69d70587494ef114e6485f4b"
server_key_version: "encryptionkey@0"
I20260619 02:48:20.456713 3287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:20.471637 3287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:20.473474 3287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:20.473625 3287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:20.477635 3287 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:45015
I20260619 02:48:20.477671 3355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:45015 every 8 connection(s)
I20260619 02:48:20.478044 3287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:48:20.479539 3356 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:48:20.480198 3356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Bootstrap starting.
I20260619 02:48:20.480942 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 3287
I20260619 02:48:20.481288 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:36223
--local_ip_for_outbound_sockets=127.19.113.65
--tserver_master_addrs=127.19.113.126:45015
--webserver_port=43365
--webserver_interface=127.19.113.65
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:48:20.488854 3356 log.cc:824] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:20.490967 3356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:48:20.491246 3356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Bootstrap complete.
I20260619 02:48:20.493439 3356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:48:20.493693 3356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3cfbd1f02540ecabc9e0abc826149c, State: Initialized, Role: FOLLOWER
I20260619 02:48:20.493842 3356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:48:20.493901 3356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:48:20.493947 3356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:48:20.493991 3356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:48:20.495121 3356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:48:20.495224 3356 leader_election.cc:302] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e3cfbd1f02540ecabc9e0abc826149c; no voters:
I20260619 02:48:20.495388 3356 leader_election.cc:288] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:48:20.495471 3360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:48:20.495654 3356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:48:20.495684 3360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 2 LEADER]: Becoming Leader. State: Replica: 0e3cfbd1f02540ecabc9e0abc826149c, State: Running, Role: LEADER
I20260619 02:48:20.495779 3360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:48:20.496147 3361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } } }
I20260619 02:48:20.496289 3360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e3cfbd1f02540ecabc9e0abc826149c. Latest consensus state: current_term: 2 leader_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } } }
I20260619 02:48:20.496390 3360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: This master's current role is: LEADER
I20260619 02:48:20.496349 3361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: This master's current role is: LEADER
I20260619 02:48:20.496685 3369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:48:20.497640 3369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:48:20.497740 3369 catalog_manager.cc:1259] Loaded cluster ID: edceb9c6a9f243fea3edea4768bfe8b7
I20260619 02:48:20.497819 3369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:48:20.499086 3369 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:48:20.499267 3369 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Loaded TSK: 0
I20260619 02:48:20.510207 3369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Generated new TSK 1
I20260619 02:48:20.510322 3369 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:48:20.569119 3358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:20.569353 3358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:20.569386 3358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:20.570739 3358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:20.570791 3358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:20.570818 3358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:48:20.572378 3358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:36223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=43365
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:20.572610 3358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:20.572846 3358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:20.578899 3383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:20.579057 3358 server_base.cc:1034] running on GCE node
W20260619 02:48:20.578893 3384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:20.579079 3386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:20.579411 3358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:20.579670 3358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:20.580860 3358 hybrid_clock.cc:648] HybridClock initialized: now 1781837300580843 us; error 31 us; skew 500 ppm
Jun 19 02:48:20 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837300, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:20.583463 3358 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:48:20.583990 3358 webserver.cc:460] Webserver started at http://127.19.113.65:43365/ using document root <none> and password file <none>
I20260619 02:48:20.584234 3358 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:20.584311 3358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:20.586068 3358 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:48:20.586812 3393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.586985 3358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:20.587080 3358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "dacbd5b5faff48068d2f3abc90508f6f"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "1b5fe70a7aa7207203e76189ada9bd68"
server_key_iv: "b288c3fa2d3bd794d1d28100389cae80"
server_key_version: "encryptionkey@0"
I20260619 02:48:20.587554 3358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:20.612545 3358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:20.615178 3358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:20.615334 3358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:20.615581 3358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:20.616094 3400 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:48:20.617698 3358 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:48:20.617762 3358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:48:20.617796 3358 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:48:20.618628 3358 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:48:20.618670 3358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:48:20.618760 3400 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Bootstrap starting.
I20260619 02:48:20.625665 3358 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:36223
I20260619 02:48:20.625903 3507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:36223 every 8 connection(s)
I20260619 02:48:20.626201 3358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:48:20.626835 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 3358
I20260619 02:48:20.627127 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:45427
--local_ip_for_outbound_sockets=127.19.113.66
--tserver_master_addrs=127.19.113.126:45015
--webserver_port=35447
--webserver_interface=127.19.113.66
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:48:20.628728 3400 log.cc:824] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:20.632103 3400 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:48:20.632506 3400 tablet_bootstrap.cc:492] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Bootstrap complete.
I20260619 02:48:20.633646 3400 ts_tablet_manager.cc:1397] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.006s
I20260619 02:48:20.635175 3400 raft_consensus.cc:348] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:20.635974 3400 raft_consensus.cc:729] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dacbd5b5faff48068d2f3abc90508f6f, State: Initialized, Role: FOLLOWER
I20260619 02:48:20.636215 3400 consensus_queue.cc:260] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } } peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } }
I20260619 02:48:20.636579 3400 ts_tablet_manager.cc:1428] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
Jun 19 02:48:20 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837300, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:48:20.640590 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.627061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:59153 (local address 127.19.113.126:45015)
0619 02:48:20.628904 (+ 1843us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:20.628914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:20.630159 (+ 1245us) server_negotiation.cc:407] Connection header received
0619 02:48:20.630300 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:20.630307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:20.630367 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:20.630519 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:20.631364 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.632719 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.634204 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.634739 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.634842 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.637420 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:20.637424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:20.637425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:20.637436 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:20.639006 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.639388 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.639391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.639391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.639480 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.639701 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.639702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.639702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.639927 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:20.640000 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:20.640280 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 02:48:20.640398 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1783,"thread_start_us":91,"threads_started":1}
I20260619 02:48:20.640590 3510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.626998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:59153)
0619 02:48:20.629109 (+ 2111us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:20.629168 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:20.629804 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:20.630591 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:20.630596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:20.630959 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:20.631223 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.631230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.632876 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.632877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.634096 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.634099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.634958 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.634959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.635080 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.635662 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:20.635672 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:20.637307 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:20.639085 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:20.639092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:20.639094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:20.639313 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:20.639540 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:20.639540 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:20.639541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:20.639645 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:20.639989 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:20.639996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:20.640131 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 02:48:20.640218 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:48:20.640395 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2066,"thread_start_us":89,"threads_started":1}
I20260619 02:48:20.641214 3508 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45015
I20260619 02:48:20.641305 3508 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:20.641466 3508 heartbeater.cc:511] Master 127.19.113.126:45015 requested a full tablet report, sending...
I20260619 02:48:20.642122 3320 ts_manager.cc:194] Registered new tserver with Master: dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223)
I20260619 02:48:20.643157 3320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:59153
I20260619 02:48:20.644032 3508 heartbeater.cc:503] Master 127.19.113.126:45015 was elected leader, sending a full tablet report...
Jun 19 02:48:20 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837300, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.65@KRBTEST.COM
I20260619 02:48:20.651328 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.642763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.19.113.65:36223)
0619 02:48:20.643591 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:20.643596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:20.643606 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:48:20.643637 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:20.643638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:20.643652 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:20.643720 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:20.644400 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.645084 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.646177 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.646456 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.646509 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.648798 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:20.648805 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:20.648806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:20.648812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:20.650154 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.650558 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.650560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.650560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.650612 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.650783 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.650783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.650784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.650946 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:20.651005 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:20.651177 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 02:48:20.651251 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":85,"threads_started":1}
I20260619 02:48:20.651351 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.642700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:36223 (local address 127.0.0.1:55510)
0619 02:48:20.642952 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:20.642983 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:20.643043 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:20.643736 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:20.643741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:20.644060 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:20.644203 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.644206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.645162 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.645163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.646095 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.646098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.646521 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.646522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.646592 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.647115 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:20.647120 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:20.648720 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:20.650228 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:20.650235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:20.650237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:20.650488 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:20.650670 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:20.650671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:20.650671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:20.650733 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:20.650995 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:20.651001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:20.651054 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 02:48:20.651134 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:48:20.651253 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":93,"threads_started":1}
I20260619 02:48:20.651713 3441 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1d6444ac804d7ba97a360016858f0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:00 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:55510
I20260619 02:48:20.652220 3517 tablet_replica.cc:330] stopping tablet replica
I20260619 02:48:20.652308 3517 raft_consensus.cc:2229] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:48:20.652374 3517 raft_consensus.cc:2258] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:20.652691 3517 ts_tablet_manager.cc:1905] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:20.654750 3517 ts_tablet_manager.cc:1918] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:20.654809 3517 log.cc:1196] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal/wals/6b1d6444ac804d7ba97a360016858f0a
I20260619 02:48:20.655047 3517 ts_tablet_manager.cc:1939] T 6b1d6444ac804d7ba97a360016858f0a P dacbd5b5faff48068d2f3abc90508f6f: Deleting consensus metadata
I20260619 02:48:20.655508 3304 catalog_manager.cc:4841] TS dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223): tablet 6b1d6444ac804d7ba97a360016858f0a (table test-table [id=a6be668316154850a4dd0dbc8d3c9115]) successfully deleted
W20260619 02:48:20.737762 3512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:20.738062 3512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:20.738098 3512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:20.739919 3512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:20.739979 3512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:20.740020 3512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:48:20.742200 3512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:45427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=35447
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:20.742466 3512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:20.742724 3512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:48:20.748113 3512 server_base.cc:1034] running on GCE node
W20260619 02:48:20.748059 3522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:20.748034 3525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:20.748059 3523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:20.748508 3512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:20.748744 3512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:20.749899 3512 hybrid_clock.cc:648] HybridClock initialized: now 1781837300749882 us; error 31 us; skew 500 ppm
Jun 19 02:48:20 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837300, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:20.752326 3512 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:48:20.752743 3512 webserver.cc:460] Webserver started at http://127.19.113.66:35447/ using document root <none> and password file <none>
I20260619 02:48:20.752945 3512 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:20.753008 3512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:20.754264 3512 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:48:20.754781 3532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.754945 3512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.755023 3512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "4bc1b6e47d9a4307950ee2a69557ef68"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "e6a4aaa33001032d22b9cb2807f63dc4"
server_key_iv: "634fb0d3981c0f821ab47a9d66818987"
server_key_version: "encryptionkey@0"
I20260619 02:48:20.755354 3512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:20.763474 3512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:20.765139 3512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:20.765280 3512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:20.765478 3512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:20.765786 3512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:20.765823 3512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.765851 3512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:20.765877 3512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.771476 3512 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:45427
I20260619 02:48:20.771530 3645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:45427 every 8 connection(s)
I20260619 02:48:20.771842 3512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:48:20.774338 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 3512
I20260619 02:48:20.774575 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:45509
--local_ip_for_outbound_sockets=127.19.113.67
--tserver_master_addrs=127.19.113.126:45015
--webserver_port=39185
--webserver_interface=127.19.113.67
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:48:20 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837300, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:48:20.782620 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.772473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:55761 (local address 127.19.113.126:45015)
0619 02:48:20.772516 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:20.772518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:20.772749 (+ 231us) server_negotiation.cc:407] Connection header received
0619 02:48:20.773202 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:20.773203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:20.773214 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:20.773260 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:20.774190 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.774845 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.775965 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.776187 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.776246 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.779806 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:20.779808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:20.779808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:20.779818 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:20.781252 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.781627 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.781628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.781629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.781693 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.781903 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.781904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.781904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.782081 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:20.782172 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:20.782423 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 02:48:20.782521 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:48:20.782683 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.772463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:55761)
0619 02:48:20.772600 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:20.772660 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:20.773123 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:20.773315 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:20.773319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:20.773883 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:20.774076 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.774080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.774951 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.774952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.775863 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.775865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.776259 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.776260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.776401 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.778107 (+ 1706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:20.778116 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:20.779691 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:20.781340 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:20.781347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:20.781349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:20.781560 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:20.781753 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:20.781753 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:20.781754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:20.781850 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:20.782175 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:20.782179 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:20.782300 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 02:48:20.782383 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:48:20.782509 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260619 02:48:20.783236 3646 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45015
I20260619 02:48:20.783351 3646 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:20.783560 3646 heartbeater.cc:511] Master 127.19.113.126:45015 requested a full tablet report, sending...
I20260619 02:48:20.783974 3320 ts_manager.cc:194] Registered new tserver with Master: 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427)
I20260619 02:48:20.784364 3320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:55761
W20260619 02:48:20.856316 3649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:20.856526 3649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:20.856549 3649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:20.857769 3649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:20.857810 3649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:20.857839 3649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:48:20.859431 3649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:45509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=39185
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:20.859679 3649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:20.859896 3649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:20.864696 3654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:20.864775 3657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:20.864859 3649 server_base.cc:1034] running on GCE node
W20260619 02:48:20.864867 3655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:20.865136 3649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:20.865351 3649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:20.866490 3649 hybrid_clock.cc:648] HybridClock initialized: now 1781837300866478 us; error 26 us; skew 500 ppm
Jun 19 02:48:20 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837300, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:20.868948 3649 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:48:20.869362 3649 webserver.cc:460] Webserver started at http://127.19.113.67:39185/ using document root <none> and password file <none>
I20260619 02:48:20.869568 3649 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:20.869642 3649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:20.871073 3649 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:48:20.871624 3664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.871774 3649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.871856 3649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "3e828485ea58447f92a413721d0e91d5"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "abc6bb6670ec1f4c53929e35cb46866c"
server_key_iv: "8577581cf53fb48e1d562bdf675a84ef"
server_key_version: "encryptionkey@0"
I20260619 02:48:20.872185 3649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:20.884881 3649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:20.887061 3649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:20.887205 3649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:20.887418 3649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:20.887718 3649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:20.887751 3649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.887780 3649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:20.887797 3649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:20.893550 3649 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:45509
I20260619 02:48:20.893625 3777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:45509 every 8 connection(s)
I20260619 02:48:20.893955 3649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:48:20.899603 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 3649
Jun 19 02:48:20 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837300, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:48:20.903987 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.894695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:36523 (local address 127.19.113.126:45015)
0619 02:48:20.894778 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:20.894781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:20.894987 (+ 206us) server_negotiation.cc:407] Connection header received
0619 02:48:20.895532 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:20.895534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:20.895546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:20.895605 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:20.896690 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.897309 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.898383 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.898575 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.898646 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.901044 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:20.901045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:20.901045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:20.901054 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:20.902392 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.902926 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.902927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.902927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.902994 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.903269 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.903269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.903270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.903399 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:20.903464 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:20.903836 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 02:48:20.903909 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:48:20.904232 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.894684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:36523)
0619 02:48:20.894843 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:20.894911 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:20.895422 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:20.895675 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:20.895678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:20.896324 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:20.896578 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.896584 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.897418 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.897419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.898302 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.898306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.898637 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.898638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.898740 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.899263 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:20.899272 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:20.900930 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:20.902473 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:20.902483 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:20.902486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:20.902835 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:20.903049 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:20.903049 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:20.903050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:20.903201 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:20.903461 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:20.903467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:20.903658 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 02:48:20.903784 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 02:48:20.903967 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260619 02:48:20.904706 3778 heartbeater.cc:347] Connected to a master server at 127.19.113.126:45015
I20260619 02:48:20.904826 3778 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:20.905025 3778 heartbeater.cc:511] Master 127.19.113.126:45015 requested a full tablet report, sending...
I20260619 02:48:20.905422 3320 ts_manager.cc:194] Registered new tserver with Master: 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509)
I20260619 02:48:20.905791 3320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:36523
I20260619 02:48:20.907367 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.900012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.19.113.126:45015)
0619 02:48:20.900136 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:20.900137 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:20.900285 (+ 148us) server_negotiation.cc:407] Connection header received
0619 02:48:20.900356 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:20.900356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:20.900365 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:20.900415 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:20.901108 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.901854 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.902723 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.902964 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.903031 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.904454 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:20.904456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:20.904456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:20.904463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:20.905765 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.906514 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.906515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.906515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.906571 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:20.906750 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:20.906750 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:20.906751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:20.906904 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:20.907044 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:20.907134 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 02:48:20.907297 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260619 02:48:20.908103 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:48:20.910591 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.908397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.19.113.126:45015)
0619 02:48:20.908469 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:20.908471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:20.908568 (+ 97us) server_negotiation.cc:407] Connection header received
0619 02:48:20.908619 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:20.908620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:20.908653 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:20.908707 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:48:20.908782 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.909240 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.909974 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.910178 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.910231 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.910356 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:48:20.910448 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:48:20.910502 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:20.910514 (+ 12us) server_negotiation.cc:299] Negotiation successful
0619 02:48:20.910531 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:48:20.910912 3320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:48:20.912582 3320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:48:20.914794 3441 tablet_service.cc:1467] Processing CreateTablet for tablet ef33688d6ed6426fb3463705cc2cb886 (DEFAULT_TABLE table=test-table [id=2504beea88f5422082c5acdb9a89dd8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:20.915011 3441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef33688d6ed6426fb3463705cc2cb886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:20.916250 3400 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Bootstrap starting.
I20260619 02:48:20.916589 3400 tablet_bootstrap.cc:654] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:20.917263 3400 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: No bootstrap required, opened a new log
I20260619 02:48:20.917318 3400 ts_tablet_manager.cc:1397] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:48:20.917662 3400 raft_consensus.cc:348] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:20.917745 3400 raft_consensus.cc:374] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:20.917802 3400 raft_consensus.cc:729] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dacbd5b5faff48068d2f3abc90508f6f, State: Initialized, Role: FOLLOWER
I20260619 02:48:20.917853 3400 consensus_queue.cc:260] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:20.917948 3400 ts_tablet_manager.cc:1428] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:48:20.918709 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.914563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.19.113.66:45427)
0619 02:48:20.914727 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:20.914732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:20.914742 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:48:20.914782 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:20.914784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:20.914801 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:20.914913 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:20.915053 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.916257 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.917605 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.918323 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.918409 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.918457 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:20.918582 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 02:48:20.918624 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260619 02:48:20.918721 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.914483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.19.113.67:45509)
0619 02:48:20.914778 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:20.914781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:20.914789 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:48:20.914823 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:20.914826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:20.914839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:20.914927 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:20.915130 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.916250 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.917463 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.918254 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.918324 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.918392 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:20.918608 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 02:48:20.918650 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":90,"threads_started":1}
I20260619 02:48:20.918708 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.914368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:45509 (local address 127.0.0.1:42762)
0619 02:48:20.914550 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:20.914561 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:20.914611 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:20.915010 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:20.915011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:20.915013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:20.915058 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.915060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.916344 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.916345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.917371 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.917374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.918346 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.918347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.918533 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.918537 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:48:20.918582 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 02:48:20.918617 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":87,"thread_start_us":106,"threads_started":1}
I20260619 02:48:20.918707 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:20.914368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:45427 (local address 127.0.0.1:57974)
0619 02:48:20.914421 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:20.914431 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:20.914469 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:20.914934 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:20.914935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:20.914937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:20.914989 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.914990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.916349 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.916350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.917506 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:20.917510 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:20.918411 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:20.918412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:20.918479 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:20.918502 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:48:20.918564 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:48:20.918617 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":46}
I20260619 02:48:20.919454 3712 tablet_service.cc:1467] Processing CreateTablet for tablet ef33688d6ed6426fb3463705cc2cb886 (DEFAULT_TABLE table=test-table [id=2504beea88f5422082c5acdb9a89dd8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:20.919456 3580 tablet_service.cc:1467] Processing CreateTablet for tablet ef33688d6ed6426fb3463705cc2cb886 (DEFAULT_TABLE table=test-table [id=2504beea88f5422082c5acdb9a89dd8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:20.919770 3580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef33688d6ed6426fb3463705cc2cb886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:20.919772 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef33688d6ed6426fb3463705cc2cb886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:20.922372 3789 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Bootstrap starting.
I20260619 02:48:20.922423 3788 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: Bootstrap starting.
I20260619 02:48:20.923115 3789 tablet_bootstrap.cc:654] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:20.923116 3788 tablet_bootstrap.cc:654] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:20.923558 3789 log.cc:824] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:20.923573 3788 log.cc:824] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:20.924213 3789 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: No bootstrap required, opened a new log
I20260619 02:48:20.924216 3788 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: No bootstrap required, opened a new log
I20260619 02:48:20.924268 3789 ts_tablet_manager.cc:1397] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:48:20.924271 3788 ts_tablet_manager.cc:1397] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:48:20.925724 3789 raft_consensus.cc:348] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:20.925755 3788 raft_consensus.cc:348] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:20.925865 3789 raft_consensus.cc:374] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:20.925889 3789 raft_consensus.cc:729] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bc1b6e47d9a4307950ee2a69557ef68, State: Initialized, Role: FOLLOWER
I20260619 02:48:20.925891 3788 raft_consensus.cc:374] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:20.925935 3788 raft_consensus.cc:729] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e828485ea58447f92a413721d0e91d5, State: Initialized, Role: FOLLOWER
I20260619 02:48:20.925990 3789 consensus_queue.cc:260] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:20.926044 3788 consensus_queue.cc:260] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:20.926273 3789 ts_tablet_manager.cc:1428] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:48:20.926283 3646 heartbeater.cc:503] Master 127.19.113.126:45015 was elected leader, sending a full tablet report...
I20260619 02:48:20.926367 3788 ts_tablet_manager.cc:1428] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:48:20.926450 3778 heartbeater.cc:503] Master 127.19.113.126:45015 was elected leader, sending a full tablet report...
W20260619 02:48:21.022338 3647 tablet.cc:2369] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:48:21.063725 3793 raft_consensus.cc:482] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:48:21.063866 3793 raft_consensus.cc:504] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:21.064246 3793 leader_election.cc:288] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427), dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223)
I20260619 02:48:21.068483 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:21.064719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:45697 (local address 127.19.113.66:45427)
0619 02:48:21.064782 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:21.064784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:21.064791 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:48:21.064815 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:21.064816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:21.064824 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:21.064871 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:21.065079 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.065852 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.067480 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.068048 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.068165 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:21.068176 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:21.068350 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 02:48:21.068393 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:48:21.068483 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:21.064634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:45059 (local address 127.19.113.65:36223)
0619 02:48:21.064705 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:21.064707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:21.064715 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:48:21.064742 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:21.064743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:21.064774 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:21.064841 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:21.065066 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.065856 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.067472 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.068114 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.068184 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:21.068264 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:21.068349 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 02:48:21.068393 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:48:21.068509 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:21.064412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:45427 (local address 127.19.113.67:45697)
0619 02:48:21.064496 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:21.064508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:21.064552 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:21.064885 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:21.064886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:21.064887 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:21.064963 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:21.064965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.065967 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.065968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:21.067386 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:21.067388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.068183 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.068185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:21.068272 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:21.068278 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:48:21.068337 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:48:21.068387 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":99}
I20260619 02:48:21.068516 3794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:21.064425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:36223 (local address 127.19.113.67:45059)
0619 02:48:21.064606 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:21.064615 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:21.064658 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:21.064859 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:21.064861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:21.064863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:21.064963 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:21.064965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.065967 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.065968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:21.067355 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:21.067359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.068195 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.068195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:21.068272 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:21.068278 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:48:21.068337 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:48:21.068384 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":57,"thread_start_us":101,"threads_started":1}
I20260619 02:48:21.068872 3462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef33688d6ed6426fb3463705cc2cb886" candidate_uuid: "3e828485ea58447f92a413721d0e91d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dacbd5b5faff48068d2f3abc90508f6f" is_pre_election: true
I20260619 02:48:21.068917 3600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef33688d6ed6426fb3463705cc2cb886" candidate_uuid: "3e828485ea58447f92a413721d0e91d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" is_pre_election: true
I20260619 02:48:21.069041 3462 raft_consensus.cc:2454] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e828485ea58447f92a413721d0e91d5 in term 0.
I20260619 02:48:21.069046 3600 raft_consensus.cc:2454] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e828485ea58447f92a413721d0e91d5 in term 0.
I20260619 02:48:21.069253 3665 leader_election.cc:302] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e828485ea58447f92a413721d0e91d5, dacbd5b5faff48068d2f3abc90508f6f; no voters:
I20260619 02:48:21.069411 3793 raft_consensus.cc:2781] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:48:21.069470 3793 raft_consensus.cc:482] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:48:21.069485 3793 raft_consensus.cc:3037] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:21.070168 3793 raft_consensus.cc:504] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:21.070293 3793 leader_election.cc:288] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [CANDIDATE]: Term 1 election: Requested vote from peers 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427), dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223)
I20260619 02:48:21.070439 3462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef33688d6ed6426fb3463705cc2cb886" candidate_uuid: "3e828485ea58447f92a413721d0e91d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dacbd5b5faff48068d2f3abc90508f6f"
I20260619 02:48:21.070462 3600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef33688d6ed6426fb3463705cc2cb886" candidate_uuid: "3e828485ea58447f92a413721d0e91d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bc1b6e47d9a4307950ee2a69557ef68"
I20260619 02:48:21.070515 3462 raft_consensus.cc:3037] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:21.070535 3600 raft_consensus.cc:3037] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:21.071306 3462 raft_consensus.cc:2454] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e828485ea58447f92a413721d0e91d5 in term 1.
I20260619 02:48:21.071381 3600 raft_consensus.cc:2454] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e828485ea58447f92a413721d0e91d5 in term 1.
I20260619 02:48:21.071542 3665 leader_election.cc:302] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e828485ea58447f92a413721d0e91d5, dacbd5b5faff48068d2f3abc90508f6f; no voters:
I20260619 02:48:21.071635 3793 raft_consensus.cc:2781] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:48:21.071810 3793 raft_consensus.cc:686] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 1 LEADER]: Becoming Leader. State: Replica: 3e828485ea58447f92a413721d0e91d5, State: Running, Role: LEADER
I20260619 02:48:21.071897 3793 consensus_queue.cc:237] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:21.072784 3319 catalog_manager.cc:5495] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e828485ea58447f92a413721d0e91d5 (127.19.113.67). New cstate: current_term: 1 leader_uuid: "3e828485ea58447f92a413721d0e91d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } health_report { overall_health: UNKNOWN } } }
I20260619 02:48:21.105115 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:21.102775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.19.113.67:45509)
0619 02:48:21.102842 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:21.102844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:21.102851 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:48:21.102914 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:21.102914 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:21.102932 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:21.102989 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:48:21.103108 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.103572 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.104301 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.104487 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.104540 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:21.104687 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:48:21.104736 (+ 49us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 02:48:21.105022 (+ 286us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42770: token expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 02:48:21.105204 3785 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42770: token expired
I20260619 02:48:21.112063 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:21.105515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.19.113.126:45015)
0619 02:48:21.105556 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:21.105557 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:21.105565 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:48:21.105585 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:21.105586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:21.105594 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:21.105642 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:21.106149 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.106602 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.107402 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.107691 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.107762 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:21.109179 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:21.109181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:21.109181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:21.109188 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:21.110906 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:21.111299 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:21.111300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:21.111300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:21.111373 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:21.111561 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:21.111562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:21.111562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:21.111675 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:21.111760 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:21.111872 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:48:21.111956 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:48:21.115192 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:21.113029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.19.113.67:45509)
0619 02:48:21.113068 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:21.113070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:21.113092 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:48:21.113133 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:21.113133 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:21.113145 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:21.113197 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:48:21.113264 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.113926 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.114567 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:21.114765 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:21.114811 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:21.114943 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:48:21.115035 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:48:21.115072 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:21.115113 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 02:48:21.115135 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:48:21.117007 3462 raft_consensus.cc:1261] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Refusing update from remote peer 3e828485ea58447f92a413721d0e91d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:48:21.117019 3600 raft_consensus.cc:1261] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 FOLLOWER]: Refusing update from remote peer 3e828485ea58447f92a413721d0e91d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:48:21.117377 3795 consensus_queue.cc:1035] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:48:21.117512 3793 consensus_queue.cc:1035] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:48:21.119657 3799 mvcc.cc:205] Tried to move back new op lower bound from 7298405585372540928 to 7298405585191202816. Current Snapshot: MvccSnapshot[applied={T|T < 7298405585372540928}]
I20260619 02:48:21.121660 3319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:48:21.121759 3319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:48:21.122404 3319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Sending DeleteTablet for 3 replicas of tablet ef33688d6ed6426fb3463705cc2cb886
I20260619 02:48:21.122695 3580 tablet_service.cc:1514] Processing DeleteTablet for tablet ef33688d6ed6426fb3463705cc2cb886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:21 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:57974
I20260619 02:48:21.122789 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 3358
I20260619 02:48:21.122830 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet ef33688d6ed6426fb3463705cc2cb886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:21 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:42762
I20260619 02:48:21.123325 3808 tablet_replica.cc:330] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: stopping tablet replica
I20260619 02:48:21.123423 3808 raft_consensus.cc:2229] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:48:21.123539 3808 raft_consensus.cc:2258] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:21.123914 3809 tablet_replica.cc:330] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: stopping tablet replica
I20260619 02:48:21.123982 3809 raft_consensus.cc:2229] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:48:21.124050 3809 raft_consensus.cc:2258] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:21.124433 3809 ts_tablet_manager.cc:1905] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:21.125005 3808 ts_tablet_manager.cc:1905] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:21.126010 3809 ts_tablet_manager.cc:1918] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:21.126080 3809 log.cc:1196] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/ef33688d6ed6426fb3463705cc2cb886
I20260619 02:48:21.126330 3809 ts_tablet_manager.cc:1939] T ef33688d6ed6426fb3463705cc2cb886 P 3e828485ea58447f92a413721d0e91d5: Deleting consensus metadata
I20260619 02:48:21.126618 3808 ts_tablet_manager.cc:1918] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:21.126690 3808 log.cc:1196] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/ef33688d6ed6426fb3463705cc2cb886
I20260619 02:48:21.126926 3304 catalog_manager.cc:4841] TS 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509): tablet ef33688d6ed6426fb3463705cc2cb886 (table test-table [id=2504beea88f5422082c5acdb9a89dd8f]) successfully deleted
I20260619 02:48:21.126984 3808 ts_tablet_manager.cc:1939] T ef33688d6ed6426fb3463705cc2cb886 P 4bc1b6e47d9a4307950ee2a69557ef68: Deleting consensus metadata
I20260619 02:48:21.127441 3305 catalog_manager.cc:4841] TS 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427): tablet ef33688d6ed6426fb3463705cc2cb886 (table test-table [id=2504beea88f5422082c5acdb9a89dd8f]) successfully deleted
W20260619 02:48:21.130246 3304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:36223 (error 108)
I20260619 02:48:21.130602 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 3512
I20260619 02:48:21.130692 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:21.130559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:36223 (local address 127.0.0.1:55516)
0619 02:48:21.130609 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:21.130658 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:36223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 02:48:21.131553 3304 catalog_manager.cc:4584] TS dacbd5b5faff48068d2f3abc90508f6f (127.19.113.65:36223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef33688d6ed6426fb3463705cc2cb886: Network error: Client connection negotiation failed: client connection to 127.19.113.65:36223: connect: Connection refused (error 111)
I20260619 02:48:21.135411 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 3649
I20260619 02:48:21.139969 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 3287
I20260619 02:48:41.144572 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45015
--webserver_interface=127.19.113.126
--webserver_port=32789
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:45015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:48:41.223501 3811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:41.223706 3811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:41.223726 3811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:41.224843 3811 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:48:41.224880 3811 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:48:41.224891 3811 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:48:41.224902 3811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:48:41.224913 3811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:41.224947 3811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:41.224969 3811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:48:41.224992 3811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:48:41.226468 3811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:45015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=32789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:41.226697 3811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:41.226922 3811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:41.232076 3816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:41.232156 3817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:41.232251 3811 server_base.cc:1034] running on GCE node
W20260619 02:48:41.232177 3819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:41.232578 3811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:41.232887 3811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:41.234031 3811 hybrid_clock.cc:648] HybridClock initialized: now 1781837321234024 us; error 39 us; skew 500 ppm
Jun 19 02:48:41 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837321, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:41.241254 3811 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:48:41.241711 3811 webserver.cc:460] Webserver started at http://127.19.113.126:32789/ using document root <none> and password file <none>
I20260619 02:48:41.241940 3811 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:41.241989 3811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:41.243407 3811 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:48:41.244031 3826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:41.244197 3811 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:41.244288 3811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "0e3cfbd1f02540ecabc9e0abc826149c"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "f5dae2ea45f90b6e5b27fc0d66187e50"
server_key_iv: "a4f9716b69d70587494ef114e6485f4b"
server_key_version: "encryptionkey@0"
I20260619 02:48:41.244624 3811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:41.254163 3811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:41.255863 3811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:41.255983 3811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:41.259595 3811 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:45015
I20260619 02:48:41.259625 3878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:45015 every 8 connection(s)
I20260619 02:48:41.259950 3811 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:48:41.261238 3879 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:48:41.261916 3879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Bootstrap starting.
I20260619 02:48:41.269455 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 3811
I20260619 02:48:41.269768 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:36223
--local_ip_for_outbound_sockets=127.19.113.65
--tserver_master_addrs=127.19.113.126:45015
--webserver_port=43365
--webserver_interface=127.19.113.65
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:48:41.270146 3879 log.cc:824] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:41.272627 3879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:48:41.272902 3879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Bootstrap complete.
I20260619 02:48:41.275976 3879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:48:41.276297 3879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3cfbd1f02540ecabc9e0abc826149c, State: Initialized, Role: FOLLOWER
I20260619 02:48:41.276470 3879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:48:41.276549 3879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:48:41.276598 3879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:48:41.276647 3879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:48:41.277995 3879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:48:41.278108 3879 leader_election.cc:302] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e3cfbd1f02540ecabc9e0abc826149c; no voters:
I20260619 02:48:41.278267 3879 leader_election.cc:288] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:48:41.278316 3883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:48:41.278486 3883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [term 3 LEADER]: Becoming Leader. State: Replica: 0e3cfbd1f02540ecabc9e0abc826149c, State: Running, Role: LEADER
I20260619 02:48:41.278530 3879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:48:41.278578 3883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } }
I20260619 02:48:41.278847 3885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e3cfbd1f02540ecabc9e0abc826149c. Latest consensus state: current_term: 3 leader_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } } }
I20260619 02:48:41.278951 3885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: This master's current role is: LEADER
I20260619 02:48:41.278839 3884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3cfbd1f02540ecabc9e0abc826149c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45015 } } }
I20260619 02:48:41.279052 3884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c [sys.catalog]: This master's current role is: LEADER
I20260619 02:48:41.279495 3889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:48:41.280452 3889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:48:41.280565 3889 catalog_manager.cc:1259] Loaded cluster ID: edceb9c6a9f243fea3edea4768bfe8b7
I20260619 02:48:41.280602 3889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:48:41.282060 3889 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:48:41.282222 3889 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Loaded TSK: 0
I20260619 02:48:41.282263 3889 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e3cfbd1f02540ecabc9e0abc826149c: Loaded TSK: 1
F20260619 02:48:41.282492 3889 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcf9256b9ad google::LogMessage::Fail() at ??:0
@ 0x7fcf9256e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fcf9256b4de google::LogMessage::Flush() at ??:0
@ 0x7fcf9256ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcf94cd8104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fcf94cd5d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fcf9a4c51cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fcf9a4c548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fcf9a4c5594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fcf9a48bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fcf9a495ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fcf944390d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fcf9442e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcf952926db start_thread at ??:0
@ 0x7fcf9106961f clone at ??:0
W20260619 02:48:41.365243 3882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:41.365442 3882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:41.365473 3882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:41.366604 3882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:41.366643 3882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:41.366676 3882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:48:41.368153 3882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:36223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=43365
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:41.368412 3882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:41.368614 3882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:41.374326 3923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:41.374333 3920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:41.374498 3921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:41.374703 3882 server_base.cc:1034] running on GCE node
I20260619 02:48:41.375044 3882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:41.375290 3882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:41.376434 3882 hybrid_clock.cc:648] HybridClock initialized: now 1781837321376414 us; error 38 us; skew 500 ppm
Jun 19 02:48:41 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837321, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:41.378759 3882 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:48:41.379204 3882 webserver.cc:460] Webserver started at http://127.19.113.65:43365/ using document root <none> and password file <none>
I20260619 02:48:41.379419 3882 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:41.379465 3882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:41.380687 3882 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:48:41.381331 3930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:41.381490 3882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:48:41.381567 3882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "dacbd5b5faff48068d2f3abc90508f6f"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "1b5fe70a7aa7207203e76189ada9bd68"
server_key_iv: "b288c3fa2d3bd794d1d28100389cae80"
server_key_version: "encryptionkey@0"
I20260619 02:48:41.381886 3882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:41.391981 3882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:41.394029 3882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:41.394162 3882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:41.394375 3882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:41.394845 3937 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:48:41.396826 3882 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:48:41.396870 3882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:48:41.396910 3882 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:48:41.397523 3882 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:48:41.397554 3882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:48:41.397648 3937 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Bootstrap starting.
I20260619 02:48:41.404054 3882 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:36223
I20260619 02:48:41.404188 4044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:36223 every 8 connection(s)
I20260619 02:48:41.404446 3882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:48:41.404985 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 3882
I20260619 02:48:41.405252 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:45427
--local_ip_for_outbound_sockets=127.19.113.66
--tserver_master_addrs=127.19.113.126:45015
--webserver_port=35447
--webserver_interface=127.19.113.66
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:48:41.410343 3937 log.cc:824] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:41.412573 3937 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:48:41.412859 3937 tablet_bootstrap.cc:492] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Bootstrap complete.
I20260619 02:48:41.413580 3937 ts_tablet_manager.cc:1397] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.008s
I20260619 02:48:41.414624 3937 raft_consensus.cc:348] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:41.414942 3937 raft_consensus.cc:729] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dacbd5b5faff48068d2f3abc90508f6f, State: Initialized, Role: FOLLOWER
I20260619 02:48:41.415108 3937 consensus_queue.cc:260] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:41.415367 3937 ts_tablet_manager.cc:1428] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f: Time spent starting tablet: real 0.002s user 0.003s sys 0.002s
W20260619 02:48:41.499882 4047 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:48:41.405123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:33791)
0619 02:48:41.405295 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.405460 (+ 165us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:41.405904 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:41.499659 (+ 93755us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
W20260619 02:48:41.500164 4045 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:45015 (0 consecutive failures): Network error: Failed to ping master at 127.19.113.126:45015: Client connection negotiation failed: client connection to 127.19.113.126:45015: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:48:41.500538 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:41.500407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:60081)
0619 02:48:41.500462 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.500512 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:48:41.500836 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:41.500766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:49205)
0619 02:48:41.500800 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.500821 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 02:48:41.500955 4045 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:48:41.517366 4048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:41.517576 4048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:41.517599 4048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:41.518751 4048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:41.518790 4048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:41.518814 4048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:48:41.520270 4048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:45427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=35447
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:41.520493 4048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:41.520697 4048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:41.525375 4055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:41.525408 4056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:41.525408 4058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:41.525585 4048 server_base.cc:1034] running on GCE node
I20260619 02:48:41.525803 4048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:41.525997 4048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:41.527125 4048 hybrid_clock.cc:648] HybridClock initialized: now 1781837321527110 us; error 30 us; skew 500 ppm
Jun 19 02:48:41 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837321, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:41.529431 4048 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:48:41.529843 4048 webserver.cc:460] Webserver started at http://127.19.113.66:35447/ using document root <none> and password file <none>
I20260619 02:48:41.530037 4048 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:41.530076 4048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:41.531461 4048 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:48:41.532002 4065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:41.532166 4048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:48:41.532243 4048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "4bc1b6e47d9a4307950ee2a69557ef68"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "e6a4aaa33001032d22b9cb2807f63dc4"
server_key_iv: "634fb0d3981c0f821ab47a9d66818987"
server_key_version: "encryptionkey@0"
I20260619 02:48:41.532590 4048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:41.541790 4048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:41.543751 4048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:41.543853 4048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:41.544052 4048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:41.544284 4048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:41.544312 4048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:41.544332 4048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:41.544344 4048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:41.549532 4048 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:45427
I20260619 02:48:41.549600 4178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:45427 every 8 connection(s)
I20260619 02:48:41.549870 4048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:48:41.550782 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:41.550461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:51317)
0619 02:48:41.550591 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.550654 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":64,"threads_started":1}
W20260619 02:48:41.551155 4179 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:45015 (0 consecutive failures): Network error: Failed to ping master at 127.19.113.126:45015: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
I20260619 02:48:41.551370 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:41.551296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:43401)
0619 02:48:41.551331 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.551355 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 02:48:41.551595 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:41.551549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:52483)
0619 02:48:41.551568 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.551584 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260619 02:48:41.551671 4179 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:48:41.554149 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 4048
I20260619 02:48:41.554381 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:45509
--local_ip_for_outbound_sockets=127.19.113.67
--tserver_master_addrs=127.19.113.126:45015
--webserver_port=39185
--webserver_interface=127.19.113.67
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:48:41.633760 4182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:41.633965 4182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:41.633987 4182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:41.635169 4182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:41.635210 4182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:41.635238 4182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:48:41.636713 4182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:45509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=39185
--tserver_master_addrs=127.19.113.126:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:41.636943 4182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:41.637144 4182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:41.641755 4187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:41.641746 4188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:41.641821 4182 server_base.cc:1034] running on GCE node
W20260619 02:48:41.641750 4190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:41.642127 4182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:41.642355 4182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:41.643484 4182 hybrid_clock.cc:648] HybridClock initialized: now 1781837321643477 us; error 25 us; skew 500 ppm
Jun 19 02:48:41 dist-test-slave-zqpb krb5kdc[2743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837321, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:41.645714 4182 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:48:41.646097 4182 webserver.cc:460] Webserver started at http://127.19.113.67:39185/ using document root <none> and password file <none>
I20260619 02:48:41.646273 4182 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:41.646310 4182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:41.647503 4182 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:48:41.648056 4197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:41.648185 4182 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:41.648269 4182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "3e828485ea58447f92a413721d0e91d5"
format_stamp: "Formatted at 2026-06-19 02:47:59 on dist-test-slave-zqpb"
server_key: "abc6bb6670ec1f4c53929e35cb46866c"
server_key_iv: "8577581cf53fb48e1d562bdf675a84ef"
server_key_version: "encryptionkey@0"
I20260619 02:48:41.648634 4182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:41.661998 4182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:41.664026 4182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:41.664141 4182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:41.664347 4182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:41.664634 4182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:41.664666 4182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:41.664686 4182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:41.664697 4182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:41.669939 4182 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:45509
I20260619 02:48:41.669991 4310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:45509 every 8 connection(s)
I20260619 02:48:41.670291 4182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:48:41.671319 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:41.670949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:41683)
0619 02:48:41.671107 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.671181 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
W20260619 02:48:41.671706 4311 heartbeater.cc:644] Failed to heartbeat to 127.19.113.126:45015 (0 consecutive failures): Network error: Failed to ping master at 127.19.113.126:45015: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
I20260619 02:48:41.671927 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:41.671851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:46117)
0619 02:48:41.671886 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.671910 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 02:48:41.672120 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:41.672064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:47183)
0619 02:48:41.672092 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:41.672109 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260619 02:48:41.672199 4311 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:48:41.678988 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 4182
I20260619 02:48:42.501538 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:42.501311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:58339)
0619 02:48:42.501440 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:42.501477 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260619 02:48:42.552134 4316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:42.551938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:59721)
0619 02:48:42.552052 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:42.552084 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260619 02:48:42.672739 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:42.672536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:54433)
0619 02:48:42.672654 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:42.672690 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260619 02:48:43.086390 4318 raft_consensus.cc:482] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:48:43.086527 4318 raft_consensus.cc:504] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:43.086845 4318 leader_election.cc:288] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427), 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509)
Jun 19 02:48:43 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837321, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.67@KRBTEST.COM
Jun 19 02:48:43 dist-test-slave-zqpb krb5kdc[2743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837321, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.66@KRBTEST.COM
I20260619 02:48:43.098320 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:43.087013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:45509 (local address 127.19.113.65:34893)
0619 02:48:43.087133 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:43.087143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:43.087200 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:43.088110 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:43.088113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:43.088629 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:43.088904 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:43.088911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:43.090512 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:43.090512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:43.091793 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:43.091796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:43.092608 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:43.092609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:43.092738 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:43.093278 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:43.093288 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:43.094857 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:43.096789 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:43.096795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:43.096797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:43.097008 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:43.097222 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:43.097223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:43.097223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:43.097325 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:43.097741 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:43.097746 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:43.097887 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 02:48:43.098026 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 02:48:43.098192 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":113,"thread_start_us":68,"threads_started":1}
I20260619 02:48:43.098336 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:43.087121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:34893 (local address 127.19.113.67:45509)
0619 02:48:43.087264 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:43.087270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:43.087298 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:48:43.087832 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:43.087835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:43.087892 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:43.088038 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:43.089020 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:43.090402 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:43.091881 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:43.092517 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:43.092619 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:43.094994 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:43.094998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:43.094999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:43.095011 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:43.096713 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:43.097084 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:43.097087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:43.097087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:43.097169 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:43.097381 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:43.097382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:43.097382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:43.097680 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:43.097759 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:43.098088 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 02:48:43.098220 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":48,"threads_started":1}
I20260619 02:48:43.098333 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:43.087110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:44689 (local address 127.19.113.66:45427)
0619 02:48:43.087254 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:43.087265 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:43.087290 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:48:43.087862 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:43.087865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:43.087904 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:43.088038 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:43.089020 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:43.090401 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:43.091881 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:43.092511 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:43.092619 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:43.095353 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:43.095357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:43.095357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:43.095367 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:43.096992 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:43.097262 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:43.097264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:43.097265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:43.097347 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:43.097502 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:43.097503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:43.097503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:43.097750 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:43.097827 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:43.098088 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 02:48:43.098220 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":51,"threads_started":1}
I20260619 02:48:43.098371 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:43.086987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:45427 (local address 127.19.113.65:44689)
0619 02:48:43.087131 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:43.087143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:43.087200 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:43.088110 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:43.088113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:43.088629 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:43.088904 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:43.088912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:43.090510 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:43.090511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:43.091794 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:43.091796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:43.092609 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:43.092610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:43.092738 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:43.093312 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:43.093315 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:43.095251 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:43.097054 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:43.097055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:43.097056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:43.097195 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:43.097407 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:43.097407 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:43.097408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:43.097454 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:43.097818 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:43.097819 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:43.097887 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 02:48:43.098026 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 02:48:43.098254 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":153,"thread_start_us":43,"threads_started":1}
W20260619 02:48:43.099434 3932 leader_election.cc:341] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427): Not found: Tablet not found: ef33688d6ed6426fb3463705cc2cb886
W20260619 02:48:43.099515 3931 leader_election.cc:341] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509): Not found: Tablet not found: ef33688d6ed6426fb3463705cc2cb886
I20260619 02:48:43.099538 3931 leader_election.cc:302] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dacbd5b5faff48068d2f3abc90508f6f; no voters: 3e828485ea58447f92a413721d0e91d5, 4bc1b6e47d9a4307950ee2a69557ef68
I20260619 02:48:43.099624 4318 raft_consensus.cc:2726] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:48:43.502053 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:43.501937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:40887)
0619 02:48:43.501979 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:43.502012 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:48:43.552719 4323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:43.552517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:39215)
0619 02:48:43.552637 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:43.552680 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260619 02:48:43.673343 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:43.673127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:48721)
0619 02:48:43.673242 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:43.673280 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260619 02:48:44.502740 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:44.502522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:51195)
0619 02:48:44.502639 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:44.502678 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260619 02:48:44.553292 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:44.553098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:34147)
0619 02:48:44.553216 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:44.553252 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260619 02:48:44.673985 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:44.673727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:42927)
0619 02:48:44.673893 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:44.673939 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260619 02:48:44.691390 4328 raft_consensus.cc:482] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:48:44.691486 4328 raft_consensus.cc:504] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:44.691655 4328 leader_election.cc:288] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427), 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509)
W20260619 02:48:44.692041 3932 leader_election.cc:341] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427): Not found: Tablet not found: ef33688d6ed6426fb3463705cc2cb886
W20260619 02:48:44.692160 3931 leader_election.cc:341] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509): Not found: Tablet not found: ef33688d6ed6426fb3463705cc2cb886
I20260619 02:48:44.692212 3931 leader_election.cc:302] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dacbd5b5faff48068d2f3abc90508f6f; no voters: 3e828485ea58447f92a413721d0e91d5, 4bc1b6e47d9a4307950ee2a69557ef68
I20260619 02:48:44.692283 4328 raft_consensus.cc:2726] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:48:45.503285 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:45.503077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:41965)
0619 02:48:45.503201 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:45.503239 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260619 02:48:45.553869 4330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:45.553651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:51169)
0619 02:48:45.553783 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:45.553817 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260619 02:48:45.674656 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:45.674448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:33939)
0619 02:48:45.674574 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:45.674613 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260619 02:48:46.503990 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:46.503738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:55971)
0619 02:48:46.503880 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:46.503928 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260619 02:48:46.554477 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:46.554262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:45183)
0619 02:48:46.554398 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:46.554436 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260619 02:48:46.675215 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:46.675021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:34865)
0619 02:48:46.675141 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:46.675173 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260619 02:48:47.504622 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:47.504401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:55235)
0619 02:48:47.504541 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:47.504576 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260619 02:48:47.555169 4336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:47.554966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:39827)
0619 02:48:47.555092 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:47.555124 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260619 02:48:47.675813 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:47.675596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:44601)
0619 02:48:47.675722 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:47.675772 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":54,"threads_started":1}
I20260619 02:48:47.739578 4338 raft_consensus.cc:482] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:48:47.739675 4338 raft_consensus.cc:504] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:47.739817 4338 leader_election.cc:288] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427), 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509)
W20260619 02:48:47.740177 3931 leader_election.cc:341] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509): Not found: Tablet not found: ef33688d6ed6426fb3463705cc2cb886
W20260619 02:48:47.740263 3932 leader_election.cc:341] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427): Not found: Tablet not found: ef33688d6ed6426fb3463705cc2cb886
I20260619 02:48:47.740332 3932 leader_election.cc:302] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dacbd5b5faff48068d2f3abc90508f6f; no voters: 3e828485ea58447f92a413721d0e91d5, 4bc1b6e47d9a4307950ee2a69557ef68
I20260619 02:48:47.740458 4338 raft_consensus.cc:2726] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:48:48.505441 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:48.505122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:52317)
0619 02:48:48.505317 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:48.505389 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":63,"threads_started":1}
I20260619 02:48:48.555840 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:48.555612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:60133)
0619 02:48:48.555753 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:48.555796 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260619 02:48:48.676473 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:48.676273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:56743)
0619 02:48:48.676391 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:48.676431 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260619 02:48:49.506148 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:49.505941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:44221)
0619 02:48:49.506067 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:49.506101 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":56,"threads_started":1}
I20260619 02:48:49.556528 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:49.556329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:43753)
0619 02:48:49.556447 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:49.556489 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260619 02:48:49.677187 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:49.676943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:35959)
0619 02:48:49.677096 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:49.677142 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260619 02:48:50.506846 4345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:50.506606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:34411)
0619 02:48:50.506735 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:50.506779 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260619 02:48:50.557163 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:50.556969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:37089)
0619 02:48:50.557081 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:50.557121 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260619 02:48:50.677877 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:50.677648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:40641)
0619 02:48:50.677793 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:50.677832 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260619 02:48:51.507503 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:51.507282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:47431)
0619 02:48:51.507422 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:51.507456 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260619 02:48:51.557788 4349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:51.557577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:47041)
0619 02:48:51.557715 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:51.557749 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260619 02:48:51.678519 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:51.678271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:53497)
0619 02:48:51.678436 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:51.678473 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260619 02:48:52.114267 4351 raft_consensus.cc:482] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:48:52.114359 4351 raft_consensus.cc:504] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b6e47d9a4307950ee2a69557ef68" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 45427 } } peers { permanent_uuid: "3e828485ea58447f92a413721d0e91d5" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 45509 } } peers { permanent_uuid: "dacbd5b5faff48068d2f3abc90508f6f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 36223 } }
I20260619 02:48:52.114530 4351 leader_election.cc:288] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427), 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509)
W20260619 02:48:52.114893 3931 leader_election.cc:341] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e828485ea58447f92a413721d0e91d5 (127.19.113.67:45509): Not found: Tablet not found: ef33688d6ed6426fb3463705cc2cb886
W20260619 02:48:52.114980 3932 leader_election.cc:341] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4bc1b6e47d9a4307950ee2a69557ef68 (127.19.113.66:45427): Not found: Tablet not found: ef33688d6ed6426fb3463705cc2cb886
I20260619 02:48:52.115015 3932 leader_election.cc:302] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dacbd5b5faff48068d2f3abc90508f6f; no voters: 3e828485ea58447f92a413721d0e91d5, 4bc1b6e47d9a4307950ee2a69557ef68
I20260619 02:48:52.115127 4351 raft_consensus.cc:2726] T ef33688d6ed6426fb3463705cc2cb886 P dacbd5b5faff48068d2f3abc90508f6f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:48:52.508159 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:52.507936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:56479)
0619 02:48:52.508056 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:52.508099 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260619 02:48:52.558413 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:52.558200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:57105)
0619 02:48:52.558332 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:52.558370 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260619 02:48:52.679304 4354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:52.679066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:33465)
0619 02:48:52.679202 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:52.679247 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260619 02:48:53.508819 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:53.508570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:57105)
0619 02:48:53.508728 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:53.508772 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":59,"threads_started":1}
I20260619 02:48:53.559113 4356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:53.558912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:36455)
0619 02:48:53.559026 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:53.559066 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260619 02:48:53.679940 4357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:53.679733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:52183)
0619 02:48:53.679852 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:53.679892 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260619 02:48:54.509420 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:54.509227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:44925)
0619 02:48:54.509346 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:54.509376 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260619 02:48:54.559832 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:54.559595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:43807)
0619 02:48:54.559735 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:54.559783 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260619 02:48:54.680672 4360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:54.680422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:36883)
0619 02:48:54.680573 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:54.680615 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260619 02:48:55.510099 4361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:55.509849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:34119)
0619 02:48:55.510011 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:55.510053 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":61,"threads_started":1}
I20260619 02:48:55.560461 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:55.560249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:50717)
0619 02:48:55.560380 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:55.560416 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260619 02:48:55.681430 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:55.681146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:41005)
0619 02:48:55.681338 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:55.681383 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260619 02:48:56.510833 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:56.510622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.65:51585)
0619 02:48:56.510747 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:56.510787 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260619 02:48:56.561122 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:56.560922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.66:47693)
0619 02:48:56.561047 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:56.561081 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260619 02:48:56.680761 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 3882
I20260619 02:48:56.682260 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:56.681996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:45015 (local address 127.19.113.67:40537)
0619 02:48:56.682172 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:56.682207 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.126:45015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":54,"threads_started":1}
I20260619 02:48:56.685060 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 4048
I20260619 02:48:56.688953 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 4182
2026-06-19T02:48:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58310 ms)
[----------] 30 tests from SecurityITest (225289 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:48:56 dist-test-slave-zqpb krb5kdc[4369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:48:56 dist-test-slave-zqpb krb5kdc[4369](info): set up 2 sockets
Jun 19 02:48:56 dist-test-slave-zqpb krb5kdc[4369](info): commencing operation
krb5kdc: starting...
W20260619 02:48:57.646483 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:48:57 dist-test-slave-zqpb krb5kdc[4369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:48:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:48:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:48:57.759387 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:32883
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:32883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:48:57.838339 4382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:57.838531 4382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:57.838551 4382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:57.839690 4382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:48:57.839730 4382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:57.839742 4382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:57.839753 4382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:48:57.839763 4382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:48:57.841148 4382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:32883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:32883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:57.841338 4382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:57.841507 4382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:57.846455 4390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:57.846505 4382 server_base.cc:1034] running on GCE node
W20260619 02:48:57.846510 4388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:57.846462 4387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:57.846908 4382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:57.847182 4382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:57.848330 4382 hybrid_clock.cc:648] HybridClock initialized: now 1781837337848302 us; error 41 us; skew 500 ppm
Jun 19 02:48:57 dist-test-slave-zqpb krb5kdc[4369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837337, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:57.850770 4382 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:48:57.851222 4382 webserver.cc:460] Webserver started at http://127.19.113.126:42673/ using document root <none> and password file <none>
I20260619 02:48:57.851418 4382 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:57.851464 4382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:57.851588 4382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:48:57.852615 4382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "b3a85202f2de48e28e2173253f9d8379"
format_stamp: "Formatted at 2026-06-19 02:48:57 on dist-test-slave-zqpb"
server_key: "23dd66cb66178c287321ac144e23043b"
server_key_iv: "f1e5a0cd5d7f79160951d54d0a27743a"
server_key_version: "encryptionkey@0"
I20260619 02:48:57.852946 4382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "b3a85202f2de48e28e2173253f9d8379"
format_stamp: "Formatted at 2026-06-19 02:48:57 on dist-test-slave-zqpb"
server_key: "23dd66cb66178c287321ac144e23043b"
server_key_iv: "f1e5a0cd5d7f79160951d54d0a27743a"
server_key_version: "encryptionkey@0"
I20260619 02:48:57.854159 4382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:48:57.854852 4397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:57.855010 4382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:57.855098 4382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "b3a85202f2de48e28e2173253f9d8379"
format_stamp: "Formatted at 2026-06-19 02:48:57 on dist-test-slave-zqpb"
server_key: "23dd66cb66178c287321ac144e23043b"
server_key_iv: "f1e5a0cd5d7f79160951d54d0a27743a"
server_key_version: "encryptionkey@0"
I20260619 02:48:57.855162 4382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:57.910125 4382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:57.912273 4382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:57.912410 4382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:57.916276 4382 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:32883
I20260619 02:48:57.916326 4449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:32883 every 8 connection(s)
I20260619 02:48:57.916627 4382 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:48:57.917477 4450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:57.919879 4450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379: Bootstrap starting.
I20260619 02:48:57.920444 4450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:57.920702 4450 log.cc:824] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:57.921365 4450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379: No bootstrap required, opened a new log
I20260619 02:48:57.922976 4450 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a85202f2de48e28e2173253f9d8379" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32883 } }
I20260619 02:48:57.923118 4450 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:57.923139 4450 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a85202f2de48e28e2173253f9d8379, State: Initialized, Role: FOLLOWER
I20260619 02:48:57.923269 4450 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a85202f2de48e28e2173253f9d8379" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32883 } }
I20260619 02:48:57.923339 4450 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:48:57.923370 4450 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:48:57.923413 4450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:57.923923 4450 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a85202f2de48e28e2173253f9d8379" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32883 } }
I20260619 02:48:57.924010 4450 leader_election.cc:302] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3a85202f2de48e28e2173253f9d8379; no voters:
I20260619 02:48:57.924165 4450 leader_election.cc:288] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:48:57.924296 4453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:48:57.924438 4450 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:48:57.924479 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 4382
I20260619 02:48:57.924566 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:48:57.924468 4453 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [term 1 LEADER]: Becoming Leader. State: Replica: b3a85202f2de48e28e2173253f9d8379, State: Running, Role: LEADER
I20260619 02:48:57.924768 19909 external_mini_cluster.cc:1428] Setting key 09f74ce14c3da602590b863e64092e11
I20260619 02:48:57.924759 4453 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a85202f2de48e28e2173253f9d8379" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32883 } }
I20260619 02:48:57.925747 4454 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3a85202f2de48e28e2173253f9d8379. Latest consensus state: current_term: 1 leader_uuid: "b3a85202f2de48e28e2173253f9d8379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a85202f2de48e28e2173253f9d8379" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32883 } } }
I20260619 02:48:57.925843 4454 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [sys.catalog]: This master's current role is: LEADER
I20260619 02:48:57.926070 4471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:48:57.926204 4455 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3a85202f2de48e28e2173253f9d8379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a85202f2de48e28e2173253f9d8379" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 32883 } } }
I20260619 02:48:57.926476 4455 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379 [sys.catalog]: This master's current role is: LEADER
I20260619 02:48:57.926924 4471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:48:57.928524 4471 catalog_manager.cc:1347] Generated new cluster ID: 1ca8386f0040445a85838086ba88f133
I20260619 02:48:57.928576 4471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:48:57 dist-test-slave-zqpb krb5kdc[4369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:48:57.936187 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:57.925452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36622 (local address 127.19.113.126:32883)
0619 02:48:57.925691 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:57.925702 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:57.925740 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:48:57.925872 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:57.925879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:57.925953 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:57.926094 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:57.926798 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:57.927971 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:57.929139 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:57.929789 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:57.929904 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:57.932603 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:57.932607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:57.932607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:57.932619 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:57.934459 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:57.934853 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:57.934856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:57.934856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:57.934987 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:57.935192 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:57.935193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:57.935193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:57.935472 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:57.935591 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:57.935813 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 02:48:57.935951 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
I20260619 02:48:57.944397 4471 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:48:57.944882 4471 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:48:57.949052 4471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379: Generated new TSK 0
I20260619 02:48:57.949211 4471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:48:57.975148 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32883
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:48:58.087626 4476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:58.087821 4476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:58.087843 4476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:58.089051 4476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:58.089093 4476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:58.089123 4476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:48:58.090544 4476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:58.090762 4476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:58.090989 4476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:58.095688 4484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:58.095660 4482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:58.095713 4476 server_base.cc:1034] running on GCE node
W20260619 02:48:58.095669 4481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:58.095983 4476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:58.096210 4476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:58.097350 4476 hybrid_clock.cc:648] HybridClock initialized: now 1781837338097341 us; error 26 us; skew 500 ppm
Jun 19 02:48:58 dist-test-slave-zqpb krb5kdc[4369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837338, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:58.099742 4476 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:48:58.100111 4476 webserver.cc:460] Webserver started at http://127.19.113.65:41347/ using document root <none> and password file <none>
I20260619 02:48:58.100322 4476 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:58.100363 4476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:58.100476 4476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:48:58.101505 4476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "ea5a5cd0eef74cee94519bd2e4c9258e"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "d1faa7b12dff8f3b0d62abfccb121812"
server_key_iv: "a0a3b40663c24371075208625aefa4c5"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.101816 4476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "ea5a5cd0eef74cee94519bd2e4c9258e"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "d1faa7b12dff8f3b0d62abfccb121812"
server_key_iv: "a0a3b40663c24371075208625aefa4c5"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.103050 4476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:48:58.103771 4491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.103904 4476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:58.103978 4476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "ea5a5cd0eef74cee94519bd2e4c9258e"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "d1faa7b12dff8f3b0d62abfccb121812"
server_key_iv: "a0a3b40663c24371075208625aefa4c5"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.104040 4476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:58.122424 4476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:58.124270 4476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:58.124418 4476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:58.124650 4476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:58.124936 4476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:58.124979 4476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.125013 4476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:58.125030 4476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.130466 4476 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:40097
I20260619 02:48:58.130537 4604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:40097 every 8 connection(s)
I20260619 02:48:58.130792 4476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:48:58 dist-test-slave-zqpb krb5kdc[4369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837338, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:48:58.139266 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.131451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:33407 (local address 127.19.113.126:32883)
0619 02:48:58.131499 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.131501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.131706 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:48:58.132160 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.132161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.132175 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.132229 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:58.133065 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.133572 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.134578 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.134786 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.134833 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.136929 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:58.136931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:58.136931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:58.136938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:58.138002 (+ 1064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:58.138373 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:58.138374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:58.138375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:58.138428 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:58.138627 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:58.138628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:58.138628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:58.138740 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:58.138796 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.139068 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.139173 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:48:58.139397 4607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.131447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32883 (local address 127.19.113.65:33407)
0619 02:48:58.131595 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.131629 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.132080 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.132280 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.132283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.132818 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:58.132998 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.133002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.133669 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.133671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.134505 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.134507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.134844 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.134844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.134980 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.135542 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:58.135552 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:58.136844 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:58.138075 (+ 1231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:58.138081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:58.138083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:58.138307 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:58.138482 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:58.138482 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:58.138483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:58.138574 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:58.138793 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:58.138799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:58.138955 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.139042 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.139196 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260619 02:48:58.139814 4605 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32883
I20260619 02:48:58.139914 4605 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:58.139999 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 4476
I20260619 02:48:58.140097 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:48:58.140120 4605 heartbeater.cc:511] Master 127.19.113.126:32883 requested a full tablet report, sending...
I20260619 02:48:58.140301 19909 external_mini_cluster.cc:1428] Setting key fbd08d9b07d5a511274881d6e1383238
I20260619 02:48:58.140702 4414 ts_manager.cc:194] Registered new tserver with Master: ea5a5cd0eef74cee94519bd2e4c9258e (127.19.113.65:40097)
I20260619 02:48:58.141374 4414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:33407
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:48:58.162514 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32883
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:48:58.240444 4610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:58.240634 4610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:58.240656 4610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:58.241770 4610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:58.241808 4610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:58.241833 4610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:48:58.243245 4610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:58.243427 4610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:58.243592 4610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:58.248219 4615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:58.248232 4616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:58.248219 4618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:58.248302 4610 server_base.cc:1034] running on GCE node
I20260619 02:48:58.248615 4610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:58.248833 4610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:58.249975 4610 hybrid_clock.cc:648] HybridClock initialized: now 1781837338249951 us; error 34 us; skew 500 ppm
Jun 19 02:48:58 dist-test-slave-zqpb krb5kdc[4369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837338, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:58.252398 4610 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:48:58.252782 4610 webserver.cc:460] Webserver started at http://127.19.113.66:36863/ using document root <none> and password file <none>
I20260619 02:48:58.252983 4610 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:58.253031 4610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:58.253149 4610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:48:58.254122 4610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "ef7e1316f618422abd0c0e0394368a4b"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "c1f9c268a4d19b2a64cb7634fc872a7a"
server_key_iv: "1315396a20fd6c5fb5e2292d379be47d"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.254442 4610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "ef7e1316f618422abd0c0e0394368a4b"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "c1f9c268a4d19b2a64cb7634fc872a7a"
server_key_iv: "1315396a20fd6c5fb5e2292d379be47d"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.255689 4610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:48:58.256481 4625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.256610 4610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:58.256690 4610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "ef7e1316f618422abd0c0e0394368a4b"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "c1f9c268a4d19b2a64cb7634fc872a7a"
server_key_iv: "1315396a20fd6c5fb5e2292d379be47d"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.256759 4610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:58.273172 4610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:58.275168 4610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:58.275301 4610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:58.275514 4610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:58.275871 4610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:58.275913 4610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.275943 4610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:58.275967 4610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.281328 4610 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:36059
I20260619 02:48:58.281379 4738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:36059 every 8 connection(s)
I20260619 02:48:58.281666 4610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:48:58.286840 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 4610
I20260619 02:48:58.286973 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:48:58.287226 19909 external_mini_cluster.cc:1428] Setting key ebd3e8428efbb1004ee15c1ed6ad0050
Jun 19 02:48:58 dist-test-slave-zqpb krb5kdc[4369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837338, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:48:58.290409 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.282307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:37895 (local address 127.19.113.126:32883)
0619 02:48:58.282355 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.282357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.282577 (+ 220us) server_negotiation.cc:407] Connection header received
0619 02:48:58.283095 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.283096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.283106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.283192 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:58.284068 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.284579 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.285520 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.285718 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.285769 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.287919 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:58.287920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:58.287920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:58.287928 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:58.289185 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:58.289589 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:58.289590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:58.289590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:58.289651 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:58.289846 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:58.289847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:58.289847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:58.289968 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:58.290037 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.290243 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.290320 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:48:58.290484 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.282299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32883 (local address 127.19.113.66:37895)
0619 02:48:58.282462 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.282498 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.283019 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.283253 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.283256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.283812 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:58.284000 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.284004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.284648 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.284649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.285448 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.285450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.285778 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.285779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.285882 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.286301 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:58.286308 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:58.287832 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:58.289276 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:58.289284 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:58.289286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:58.289519 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:58.289704 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:58.289705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:58.289705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:58.289799 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:58.290024 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:58.290027 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:58.290132 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.290206 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.290321 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260619 02:48:58.290930 4739 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32883
I20260619 02:48:58.291024 4739 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:58.291188 4739 heartbeater.cc:511] Master 127.19.113.126:32883 requested a full tablet report, sending...
I20260619 02:48:58.291576 4414 ts_manager.cc:194] Registered new tserver with Master: ef7e1316f618422abd0c0e0394368a4b (127.19.113.66:36059)
I20260619 02:48:58.292068 4414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:37895
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:48:58.309463 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32883
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:48:58.387890 4744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:58.388114 4744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:58.388145 4744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:58.389251 4744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:58.389289 4744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:58.389324 4744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:48:58.390725 4744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:32883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:58.390959 4744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:58.391171 4744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:58.395680 4750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:58.395691 4752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:58.395715 4749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:58.395838 4744 server_base.cc:1034] running on GCE node
I20260619 02:48:58.396055 4744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:58.396266 4744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:58.397423 4744 hybrid_clock.cc:648] HybridClock initialized: now 1781837338397410 us; error 29 us; skew 500 ppm
Jun 19 02:48:58 dist-test-slave-zqpb krb5kdc[4369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837338, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:48:58.399739 4744 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:48:58.400122 4744 webserver.cc:460] Webserver started at http://127.19.113.67:37691/ using document root <none> and password file <none>
I20260619 02:48:58.400313 4744 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:58.400359 4744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:58.400482 4744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:48:58.401437 4744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "561352fd8edc40899a894ca828d4d2e1"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "dbfff63ba33a94218dba76c5e7699499"
server_key_iv: "7505b303ea41fea5170b6f35ced15ebb"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.401754 4744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "561352fd8edc40899a894ca828d4d2e1"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "dbfff63ba33a94218dba76c5e7699499"
server_key_iv: "7505b303ea41fea5170b6f35ced15ebb"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.402943 4744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:48:58.403697 4759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.403869 4744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:58.403951 4744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "561352fd8edc40899a894ca828d4d2e1"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "dbfff63ba33a94218dba76c5e7699499"
server_key_iv: "7505b303ea41fea5170b6f35ced15ebb"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.404016 4744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:58.427699 4744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:58.429411 4744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:58.429538 4744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:58.429749 4744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:58.430025 4744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:58.430056 4744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.430084 4744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:58.430109 4744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.435549 4744 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:39143
I20260619 02:48:58.435616 4872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:39143 every 8 connection(s)
I20260619 02:48:58.435891 4744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:48:58 dist-test-slave-zqpb krb5kdc[4369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837338, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:48:58.443732 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 4744
I20260619 02:48:58.443818 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:48:58.444005 19909 external_mini_cluster.cc:1428] Setting key f1d5dc118910be0ba7905cefcd43beb3
I20260619 02:48:58.444859 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.436569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:41713 (local address 127.19.113.126:32883)
0619 02:48:58.436632 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.436634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.436798 (+ 164us) server_negotiation.cc:407] Connection header received
0619 02:48:58.437355 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.437356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.437369 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.437418 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:58.438314 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.438972 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.439924 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.440123 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.440176 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.442253 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:58.442254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:58.442255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:58.442262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:58.443462 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:58.443858 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:58.443859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:58.443859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:58.443924 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:58.444151 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:58.444152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:58.444152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:58.444287 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:58.444353 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.444642 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.444750 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:48:58.445106 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.436529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:32883 (local address 127.19.113.67:41713)
0619 02:48:58.436681 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.436723 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.437254 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.437485 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.437490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.438032 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:58.438225 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.438230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.439046 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.439047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.439843 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.439846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.440194 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.440195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.440305 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.440803 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:48:58.440812 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:58.442163 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:58.443530 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:58.443536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:58.443537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:58.443750 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:58.443983 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:48:58.443984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:48:58.443984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:48:58.444086 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:48:58.444352 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:58.444358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:58.444514 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.444622 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.444857 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260619 02:48:58.445502 4873 heartbeater.cc:347] Connected to a master server at 127.19.113.126:32883
I20260619 02:48:58.445595 4873 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:58.445760 4873 heartbeater.cc:511] Master 127.19.113.126:32883 requested a full tablet report, sending...
I20260619 02:48:58.446125 4414 ts_manager.cc:194] Registered new tserver with Master: 561352fd8edc40899a894ca828d4d2e1 (127.19.113.67:39143)
I20260619 02:48:58.446534 4414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:41713
I20260619 02:48:58.455516 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:48:58.461916 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.456715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.19.113.126:32883)
0619 02:48:58.456787 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.456788 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.456836 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:48:58.456871 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.456872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.456880 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.456922 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:58.457444 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.457948 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.458634 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.458852 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.458933 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.459993 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:58.459994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:58.459994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:48:58.460000 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:58.461080 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:58.461371 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:58.461372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:58.461372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:58.461415 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:48:58.461536 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:48:58.461537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:48:58.461537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:48:58.461659 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:58.461730 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.461799 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.461852 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:48:58.463411 4414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:48:58.464920 4414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:48:58.470731 4885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.467008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:39143 (local address 127.0.0.1:32894)
0619 02:48:58.467168 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.467207 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.467273 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.467543 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.467547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.467549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:58.467618 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.467620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.468619 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.468620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.469585 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.469587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.470449 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.470450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.470532 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.470545 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.470602 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.470654 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":55,"threads_started":1}
I20260619 02:48:58.470743 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.467164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32894 (local address 127.19.113.67:39143)
0619 02:48:58.467375 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.467380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.467390 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:48:58.467431 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.467433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.467448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.467525 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:58.467701 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.468534 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.469698 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.470388 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.470444 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.470502 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.470623 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.470670 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
I20260619 02:48:58.471082 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.467010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:40097 (local address 127.0.0.1:35752)
0619 02:48:58.467438 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.467447 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.467497 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.467653 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.467654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.467656 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:58.467816 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.467819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.469128 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.469130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.470009 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.470011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.470831 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.470832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.470925 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.470929 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.470975 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.471011 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":22,"thread_start_us":77,"threads_started":1}
I20260619 02:48:58.471400 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.467023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:36059 (local address 127.0.0.1:40368)
0619 02:48:58.467571 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.467579 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.467653 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.467955 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.467956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.467958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:58.468001 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.468003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.469209 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.469210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.470270 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.470273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.471188 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.471189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.471246 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.471249 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.471293 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.471332 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":79,"threads_started":1}
I20260619 02:48:58.471485 4888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.467198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.19.113.65:40097)
0619 02:48:58.467389 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.467394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.467498 (+ 104us) server_negotiation.cc:407] Connection header received
0619 02:48:58.467568 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.467570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.467586 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.467677 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:58.467897 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.469016 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.470079 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.470684 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.470749 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.470797 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.471327 (+ 530us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.471407 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":80,"threads_started":1}
I20260619 02:48:58.471609 4889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.467290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.19.113.66:36059)
0619 02:48:58.467441 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.467445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.467736 (+ 291us) server_negotiation.cc:407] Connection header received
0619 02:48:58.467779 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.467781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.467796 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.467870 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:58.468073 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.469108 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.470351 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.471039 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.471090 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.471142 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.471495 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.471539 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260619 02:48:58.471500 4807 tablet_service.cc:1467] Processing CreateTablet for tablet d5b98054cce44e27b5baba9863957d01 (DEFAULT_TABLE table=test-table [id=22562338edd34c95baaa3d7dd727e5ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:58.471884 4807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b98054cce44e27b5baba9863957d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:58.472271 4673 tablet_service.cc:1467] Processing CreateTablet for tablet d5b98054cce44e27b5baba9863957d01 (DEFAULT_TABLE table=test-table [id=22562338edd34c95baaa3d7dd727e5ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:58.472508 4673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b98054cce44e27b5baba9863957d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:58.474422 4890 tablet_bootstrap.cc:492] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: Bootstrap starting.
I20260619 02:48:58.474843 4891 tablet_bootstrap.cc:492] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Bootstrap starting.
I20260619 02:48:58.474931 4539 tablet_service.cc:1467] Processing CreateTablet for tablet d5b98054cce44e27b5baba9863957d01 (DEFAULT_TABLE table=test-table [id=22562338edd34c95baaa3d7dd727e5ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:58.475271 4539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b98054cce44e27b5baba9863957d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:58.475426 4890 tablet_bootstrap.cc:654] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:58.475508 4891 tablet_bootstrap.cc:654] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:58.475873 4890 log.cc:824] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:58.476049 4891 log.cc:824] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:58.476547 4890 tablet_bootstrap.cc:492] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: No bootstrap required, opened a new log
I20260619 02:48:58.476605 4890 ts_tablet_manager.cc:1397] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:48:58.476754 4891 tablet_bootstrap.cc:492] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: No bootstrap required, opened a new log
I20260619 02:48:58.476826 4891 ts_tablet_manager.cc:1397] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:48:58.477973 4890 raft_consensus.cc:348] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.478266 4890 raft_consensus.cc:374] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:58.478367 4890 raft_consensus.cc:729] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561352fd8edc40899a894ca828d4d2e1, State: Initialized, Role: FOLLOWER
I20260619 02:48:58.478363 4894 tablet_bootstrap.cc:492] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e: Bootstrap starting.
I20260619 02:48:58.478505 4890 consensus_queue.cc:260] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.478771 4890 ts_tablet_manager.cc:1428] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:48:58.478921 4873 heartbeater.cc:503] Master 127.19.113.126:32883 was elected leader, sending a full tablet report...
I20260619 02:48:58.478988 4894 tablet_bootstrap.cc:654] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:58.478930 4891 raft_consensus.cc:348] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.479100 4891 raft_consensus.cc:374] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:58.479141 4891 raft_consensus.cc:729] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef7e1316f618422abd0c0e0394368a4b, State: Initialized, Role: FOLLOWER
I20260619 02:48:58.479249 4891 consensus_queue.cc:260] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.479517 4891 ts_tablet_manager.cc:1428] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:48:58.479563 4739 heartbeater.cc:503] Master 127.19.113.126:32883 was elected leader, sending a full tablet report...
I20260619 02:48:58.479626 4894 log.cc:824] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:58.480270 4894 tablet_bootstrap.cc:492] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e: No bootstrap required, opened a new log
I20260619 02:48:58.480326 4894 ts_tablet_manager.cc:1397] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:48:58.481608 4894 raft_consensus.cc:348] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.481734 4894 raft_consensus.cc:374] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:58.481765 4894 raft_consensus.cc:729] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea5a5cd0eef74cee94519bd2e4c9258e, State: Initialized, Role: FOLLOWER
I20260619 02:48:58.481865 4894 consensus_queue.cc:260] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.482101 4894 ts_tablet_manager.cc:1428] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:48:58.482156 4605 heartbeater.cc:503] Master 127.19.113.126:32883 was elected leader, sending a full tablet report...
W20260619 02:48:58.532214 4740 tablet.cc:2369] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:48:58.631512 4606 tablet.cc:2369] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:48:58.634785 4896 raft_consensus.cc:482] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:48:58.634905 4896 raft_consensus.cc:504] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.635316 4896 leader_election.cc:288] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 561352fd8edc40899a894ca828d4d2e1 (127.19.113.67:39143), ea5a5cd0eef74cee94519bd2e4c9258e (127.19.113.65:40097)
I20260619 02:48:58.638033 4888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.635645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:53891 (local address 127.19.113.65:40097)
0619 02:48:58.635706 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.635708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.635716 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:48:58.635745 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.635746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.635759 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.635814 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:58.635976 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.636436 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.637349 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.637721 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.637783 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.637792 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.637933 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.637969 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:48:58.638059 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.635752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:47263 (local address 127.19.113.67:39143)
0619 02:48:58.635842 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.635844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.635853 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:48:58.635877 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.635878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.635885 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.635928 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:58.636068 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.636519 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.637398 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.637762 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.637822 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.637830 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.637959 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.637996 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:48:58.638055 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.635521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:40097 (local address 127.19.113.66:53891)
0619 02:48:58.635585 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.635597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.635635 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.635843 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.635844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.635846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:58.635914 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.635915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.636512 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.636513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.637262 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.637263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.637796 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.637797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.637871 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.637874 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.637918 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.637956 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 02:48:58.638075 4899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.635528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:39143 (local address 127.19.113.66:47263)
0619 02:48:58.635735 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.635746 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.635791 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.635940 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.635941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.635942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:58.636011 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.636013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.636592 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.636594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.637313 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.637315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.637832 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.637833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.637899 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.637902 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.637945 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.637983 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":46,"thread_start_us":113,"threads_started":1}
I20260619 02:48:58.638433 4559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b98054cce44e27b5baba9863957d01" candidate_uuid: "ef7e1316f618422abd0c0e0394368a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" is_pre_election: true
I20260619 02:48:58.638554 4827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b98054cce44e27b5baba9863957d01" candidate_uuid: "ef7e1316f618422abd0c0e0394368a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561352fd8edc40899a894ca828d4d2e1" is_pre_election: true
I20260619 02:48:58.638594 4559 raft_consensus.cc:2454] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef7e1316f618422abd0c0e0394368a4b in term 0.
I20260619 02:48:58.638700 4827 raft_consensus.cc:2454] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef7e1316f618422abd0c0e0394368a4b in term 0.
I20260619 02:48:58.638813 4629 leader_election.cc:302] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea5a5cd0eef74cee94519bd2e4c9258e, ef7e1316f618422abd0c0e0394368a4b; no voters:
I20260619 02:48:58.638998 4896 raft_consensus.cc:2781] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:48:58.639043 4896 raft_consensus.cc:482] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:48:58.639067 4896 raft_consensus.cc:3037] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:58.639768 4896 raft_consensus.cc:504] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.639894 4896 leader_election.cc:288] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [CANDIDATE]: Term 1 election: Requested vote from peers 561352fd8edc40899a894ca828d4d2e1 (127.19.113.67:39143), ea5a5cd0eef74cee94519bd2e4c9258e (127.19.113.65:40097)
I20260619 02:48:58.640028 4559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b98054cce44e27b5baba9863957d01" candidate_uuid: "ef7e1316f618422abd0c0e0394368a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e"
I20260619 02:48:58.640093 4827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b98054cce44e27b5baba9863957d01" candidate_uuid: "ef7e1316f618422abd0c0e0394368a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561352fd8edc40899a894ca828d4d2e1"
I20260619 02:48:58.640111 4559 raft_consensus.cc:3037] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:58.640170 4827 raft_consensus.cc:3037] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:58.640801 4559 raft_consensus.cc:2454] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef7e1316f618422abd0c0e0394368a4b in term 1.
I20260619 02:48:58.640885 4827 raft_consensus.cc:2454] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef7e1316f618422abd0c0e0394368a4b in term 1.
I20260619 02:48:58.640961 4629 leader_election.cc:302] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea5a5cd0eef74cee94519bd2e4c9258e, ef7e1316f618422abd0c0e0394368a4b; no voters:
I20260619 02:48:58.641038 4896 raft_consensus.cc:2781] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:48:58.641212 4896 raft_consensus.cc:686] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 1 LEADER]: Becoming Leader. State: Replica: ef7e1316f618422abd0c0e0394368a4b, State: Running, Role: LEADER
I20260619 02:48:58.641314 4896 consensus_queue.cc:237] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } }
I20260619 02:48:58.642119 4412 catalog_manager.cc:5495] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b reported cstate change: term changed from 0 to 1, leader changed from <none> to ef7e1316f618422abd0c0e0394368a4b (127.19.113.66). New cstate: current_term: 1 leader_uuid: "ef7e1316f618422abd0c0e0394368a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef7e1316f618422abd0c0e0394368a4b" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 36059 } health_report { overall_health: HEALTHY } } }
I20260619 02:48:58.649755 4889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.647736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40382 (local address 127.19.113.66:36059)
0619 02:48:58.647774 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.647776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.647785 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:48:58.647812 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.647812 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.647827 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.647879 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:48:58.647956 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.648421 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.649081 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.649268 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.649312 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.649479 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:48:58.649566 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:48:58.649614 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.649661 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.649678 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:48:58.651414 4559 raft_consensus.cc:1261] T d5b98054cce44e27b5baba9863957d01 P ea5a5cd0eef74cee94519bd2e4c9258e [term 1 FOLLOWER]: Refusing update from remote peer ef7e1316f618422abd0c0e0394368a4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:48:58.651435 4827 raft_consensus.cc:1261] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 1 FOLLOWER]: Refusing update from remote peer ef7e1316f618422abd0c0e0394368a4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:48:58.651729 4900 consensus_queue.cc:1035] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "561352fd8edc40899a894ca828d4d2e1" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:48:58.651825 4896 consensus_queue.cc:1035] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea5a5cd0eef74cee94519bd2e4c9258e" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 40097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:48:58.654110 4905 mvcc.cc:205] Tried to move back new op lower bound from 7298405739113582592 to 7298405739075268608. Current Snapshot: MvccSnapshot[applied={T|T < 7298405739113582592}]
I20260619 02:48:58.654623 4904 mvcc.cc:205] Tried to move back new op lower bound from 7298405739113582592 to 7298405739075268608. Current Snapshot: MvccSnapshot[applied={T|T < 7298405739113582592}]
I20260619 02:48:58.656002 4413 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:48:58.656095 4413 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:48:58.656738 4413 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3a85202f2de48e28e2173253f9d8379: Sending DeleteTablet for 3 replicas of tablet d5b98054cce44e27b5baba9863957d01
I20260619 02:48:58.657058 4673 tablet_service.cc:1514] Processing DeleteTablet for tablet d5b98054cce44e27b5baba9863957d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:58 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:40368
I20260619 02:48:58.657061 4807 tablet_service.cc:1514] Processing DeleteTablet for tablet d5b98054cce44e27b5baba9863957d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:58 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:32894
I20260619 02:48:58.657157 4539 tablet_service.cc:1514] Processing DeleteTablet for tablet d5b98054cce44e27b5baba9863957d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:58 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:35752
I20260619 02:48:58.657310 4913 tablet_replica.cc:330] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: stopping tablet replica
I20260619 02:48:58.657406 4913 raft_consensus.cc:2229] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:48:58.657435 4914 tablet_replica.cc:330] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: stopping tablet replica
I20260619 02:48:58.657497 4913 raft_consensus.cc:2258] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:58.657624 4914 raft_consensus.cc:2229] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:48:58.657670 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 4476
I20260619 02:48:58.657917 4913 ts_tablet_manager.cc:1905] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:58.659638 4913 ts_tablet_manager.cc:1918] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:58.659699 4913 log.cc:1196] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/d5b98054cce44e27b5baba9863957d01
I20260619 02:48:58.659905 4914 raft_consensus.cc:2258] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:58.659940 4913 ts_tablet_manager.cc:1939] T d5b98054cce44e27b5baba9863957d01 P ef7e1316f618422abd0c0e0394368a4b: Deleting consensus metadata
I20260619 02:48:58.660317 4398 catalog_manager.cc:4841] TS ef7e1316f618422abd0c0e0394368a4b (127.19.113.66:36059): tablet d5b98054cce44e27b5baba9863957d01 (table test-table [id=22562338edd34c95baaa3d7dd727e5ce]) successfully deleted
I20260619 02:48:58.660485 4914 ts_tablet_manager.cc:1905] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:58.661916 4914 ts_tablet_manager.cc:1918] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:58.661983 4914 log.cc:1196] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/d5b98054cce44e27b5baba9863957d01
I20260619 02:48:58.662221 4914 ts_tablet_manager.cc:1939] T d5b98054cce44e27b5baba9863957d01 P 561352fd8edc40899a894ca828d4d2e1: Deleting consensus metadata
I20260619 02:48:58.662564 4400 catalog_manager.cc:4841] TS 561352fd8edc40899a894ca828d4d2e1 (127.19.113.67:39143): tablet d5b98054cce44e27b5baba9863957d01 (table test-table [id=22562338edd34c95baaa3d7dd727e5ce]) successfully deleted
W20260619 02:48:58.663324 4401 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:40097 (error 108)
I20260619 02:48:58.663625 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 4610
I20260619 02:48:58.664176 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.663718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:40097 (local address 127.0.0.1:35764)
0619 02:48:58.664096 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.664134 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:40097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362}
W20260619 02:48:58.665290 4401 catalog_manager.cc:4584] TS ea5a5cd0eef74cee94519bd2e4c9258e (127.19.113.65:40097): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5b98054cce44e27b5baba9863957d01: Network error: Client connection negotiation failed: client connection to 127.19.113.65:40097: connect: Connection refused (error 111)
I20260619 02:48:58.668112 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 4744
I20260619 02:48:58.672283 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 4382
2026-06-19T02:48:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1982 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T02:48:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:48:58Z Disabled control of system clock
I20260619 02:48:58.698120 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:34293
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:35579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:34293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:48:58.776552 4920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:58.776763 4920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:58.776783 4920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:58.777855 4920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:48:58.777887 4920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:58.777899 4920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:58.777910 4920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:48:58.777925 4920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:48:58.779230 4920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:34293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:34293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:58.779418 4920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:58.779649 4920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:58.784040 4928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:58.784062 4925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:58.784036 4926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:58.784189 4920 server_base.cc:1034] running on GCE node
I20260619 02:48:58.784477 4920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:58.784763 4920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:58.785902 4920 hybrid_clock.cc:648] HybridClock initialized: now 1781837338785889 us; error 27 us; skew 500 ppm
I20260619 02:48:58.787045 4920 webserver.cc:460] Webserver started at http://127.19.113.126:37221/ using document root <none> and password file <none>
I20260619 02:48:58.787266 4920 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:58.787312 4920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:58.787436 4920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:48:58.788441 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "fc5613779ada4f6ea32cc7ed2410d68e"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "3d2da34aafc863328998b83f5acd1e13"
server_key_iv: "b67bea0ce14c14f8be4a1f5e864861da"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.788769 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "fc5613779ada4f6ea32cc7ed2410d68e"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "3d2da34aafc863328998b83f5acd1e13"
server_key_iv: "b67bea0ce14c14f8be4a1f5e864861da"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.789922 4920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:48:58.790535 4934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.790684 4920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:58.790760 4920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "fc5613779ada4f6ea32cc7ed2410d68e"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "3d2da34aafc863328998b83f5acd1e13"
server_key_iv: "b67bea0ce14c14f8be4a1f5e864861da"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.790827 4920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:58.804322 4920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:58.804569 4920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:58.804692 4920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:58.808486 4920 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:34293
I20260619 02:48:58.808521 4986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:34293 every 8 connection(s)
I20260619 02:48:58.808821 4920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:48:58.809710 4987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:58.811909 4987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e: Bootstrap starting.
I20260619 02:48:58.812431 4987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:58.812703 4987 log.cc:824] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:58.813308 4987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e: No bootstrap required, opened a new log
I20260619 02:48:58.814333 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 4920
I20260619 02:48:58.814428 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:48:58.814610 19909 external_mini_cluster.cc:1428] Setting key 1707896085e24918a3b2921570e73439
I20260619 02:48:58.815385 4987 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5613779ada4f6ea32cc7ed2410d68e" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34293 } }
I20260619 02:48:58.815523 4987 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:58.815544 4987 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc5613779ada4f6ea32cc7ed2410d68e, State: Initialized, Role: FOLLOWER
I20260619 02:48:58.815654 4987 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5613779ada4f6ea32cc7ed2410d68e" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34293 } }
I20260619 02:48:58.815728 4987 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:48:58.815768 4987 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:48:58.815806 4987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:58.816342 4987 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5613779ada4f6ea32cc7ed2410d68e" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34293 } }
I20260619 02:48:58.816422 4987 leader_election.cc:302] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc5613779ada4f6ea32cc7ed2410d68e; no voters:
I20260619 02:48:58.816562 4987 leader_election.cc:288] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:48:58.816619 4992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:48:58.816790 4992 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [term 1 LEADER]: Becoming Leader. State: Replica: fc5613779ada4f6ea32cc7ed2410d68e, State: Running, Role: LEADER
I20260619 02:48:58.816799 4987 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:48:58.816898 4992 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5613779ada4f6ea32cc7ed2410d68e" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34293 } }
I20260619 02:48:58.818002 4993 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc5613779ada4f6ea32cc7ed2410d68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5613779ada4f6ea32cc7ed2410d68e" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34293 } } }
I20260619 02:48:58.818084 4993 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [sys.catalog]: This master's current role is: LEADER
I20260619 02:48:58.818231 4994 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc5613779ada4f6ea32cc7ed2410d68e. Latest consensus state: current_term: 1 leader_uuid: "fc5613779ada4f6ea32cc7ed2410d68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5613779ada4f6ea32cc7ed2410d68e" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 34293 } } }
I20260619 02:48:58.818313 5008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:48:58.818303 4994 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e [sys.catalog]: This master's current role is: LEADER
I20260619 02:48:58.818987 5008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:48:58.820603 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.815201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.19.113.126:34293)
0619 02:48:58.815419 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.815425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.815448 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:48:58.815538 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.815544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.815619 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.815751 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:58.815970 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.817507 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.818632 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.819219 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.819339 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.819965 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:58.819969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:58.819969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:48:58.820175 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:58.820215 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:48:58.820226 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:58.820284 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.820358 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.820434 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260619 02:48:58.827118 5008 catalog_manager.cc:1347] Generated new cluster ID: 3c525179e0cd473ebd6840ba3a9b2ed6
I20260619 02:48:58.827167 5008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:48:58.839912 5008 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:48:58.840580 5008 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:48:58.843477 5008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e: Generated new TSK 0
I20260619 02:48:58.843636 5008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:48:58.848482 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:34293
--builtin_ntp_servers=127.19.113.84:35579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:48:58.926275 5011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:58.926481 5011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:58.926510 5011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:58.927691 5011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:58.927727 5011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:58.927760 5011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:48:58.929083 5011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:34293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:58.929311 5011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:58.929577 5011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:58.935256 5019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:58.935259 5017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:58.935357 5011 server_base.cc:1034] running on GCE node
W20260619 02:48:58.935474 5016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:58.935714 5011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:58.935930 5011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:58.937073 5011 hybrid_clock.cc:648] HybridClock initialized: now 1781837338937060 us; error 29 us; skew 500 ppm
I20260619 02:48:58.938293 5011 webserver.cc:460] Webserver started at http://127.19.113.65:42845/ using document root <none> and password file <none>
I20260619 02:48:58.938540 5011 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:58.938583 5011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:58.938719 5011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:48:58.939807 5011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "5754f51f350d4452b64919a8f8b9850f"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "111b7f4764fdde59cd4739ca953a9f37"
server_key_iv: "d2357a79a7f1554916deb943d5eab22b"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.940127 5011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "5754f51f350d4452b64919a8f8b9850f"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "111b7f4764fdde59cd4739ca953a9f37"
server_key_iv: "d2357a79a7f1554916deb943d5eab22b"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.941304 5011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:48:58.941948 5025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.942111 5011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:58.942180 5011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "5754f51f350d4452b64919a8f8b9850f"
format_stamp: "Formatted at 2026-06-19 02:48:58 on dist-test-slave-zqpb"
server_key: "111b7f4764fdde59cd4739ca953a9f37"
server_key_iv: "d2357a79a7f1554916deb943d5eab22b"
server_key_version: "encryptionkey@0"
I20260619 02:48:58.942247 5011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:58.953033 5011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:58.953286 5011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:58.953408 5011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:58.953603 5011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:58.953892 5011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:58.953924 5011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.953950 5011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:58.953976 5011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:58.959378 5011 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:33655
I20260619 02:48:58.959431 5138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:33655 every 8 connection(s)
I20260619 02:48:58.959728 5011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:48:58.962904 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 5011
I20260619 02:48:58.962996 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:48:58.963164 19909 external_mini_cluster.cc:1428] Setting key 3b31556d4ed7f473e76d13e0bf10b51d
I20260619 02:48:58.964148 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:34293
--builtin_ntp_servers=127.19.113.84:35579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:48:58.964838 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.960379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:48841 (local address 127.19.113.126:34293)
0619 02:48:58.960423 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:58.960425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:58.960647 (+ 222us) server_negotiation.cc:407] Connection header received
0619 02:48:58.961136 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:58.961137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:58.961153 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:58.961199 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:58.961852 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.962349 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.963412 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.963671 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.963736 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.964615 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:58.964616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:58.964617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:48:58.964621 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:58.964636 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:48:58.964640 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:58.964686 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:58.964754 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:48:58.964780 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:48:58.965143 5141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:58.960363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:34293 (local address 127.19.113.65:48841)
0619 02:48:58.960531 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:58.960563 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:58.961044 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:58.961276 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:58.961280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:58.961290 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:58.961768 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.961772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.962431 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.962431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.963339 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:58.963342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:58.963757 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:58.963758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:58.963902 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:58.964469 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:48:58.964470 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:58.964509 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:48:58.964538 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:58.964687 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:58.964690 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:58.964693 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:58.964759 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 02:48:58.964858 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260619 02:48:58.965611 5139 heartbeater.cc:347] Connected to a master server at 127.19.113.126:34293
I20260619 02:48:58.965704 5139 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:58.965894 5139 heartbeater.cc:511] Master 127.19.113.126:34293 requested a full tablet report, sending...
I20260619 02:48:58.966394 4951 ts_manager.cc:194] Registered new tserver with Master: 5754f51f350d4452b64919a8f8b9850f (127.19.113.65:33655)
I20260619 02:48:58.967056 4951 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.65:48841
W20260619 02:48:59.042569 5142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:59.042760 5142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:59.042779 5142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:59.043906 5142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:59.043944 5142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:59.043969 5142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:48:59.045327 5142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:34293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:59.045531 5142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:59.045812 5142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:48:59.050468 5150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:59.050477 5147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:59.050523 5142 server_base.cc:1034] running on GCE node
W20260619 02:48:59.050477 5148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:59.050859 5142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:59.051110 5142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:59.052240 5142 hybrid_clock.cc:648] HybridClock initialized: now 1781837339052231 us; error 26 us; skew 500 ppm
I20260619 02:48:59.059145 5142 webserver.cc:460] Webserver started at http://127.19.113.66:45239/ using document root <none> and password file <none>
I20260619 02:48:59.059404 5142 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:59.059453 5142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:59.059578 5142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:48:59.060624 5142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "8215745235bd4a36b9eb1ec12e35041a"
format_stamp: "Formatted at 2026-06-19 02:48:59 on dist-test-slave-zqpb"
server_key: "04131cb5e9cfeea1b978de14f660ddc1"
server_key_iv: "ecb272948e3e6ecf15dd4bd590a499b8"
server_key_version: "encryptionkey@0"
I20260619 02:48:59.060949 5142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "8215745235bd4a36b9eb1ec12e35041a"
format_stamp: "Formatted at 2026-06-19 02:48:59 on dist-test-slave-zqpb"
server_key: "04131cb5e9cfeea1b978de14f660ddc1"
server_key_iv: "ecb272948e3e6ecf15dd4bd590a499b8"
server_key_version: "encryptionkey@0"
I20260619 02:48:59.062158 5142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:48:59.062779 5156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:59.062937 5142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:48:59.063019 5142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "8215745235bd4a36b9eb1ec12e35041a"
format_stamp: "Formatted at 2026-06-19 02:48:59 on dist-test-slave-zqpb"
server_key: "04131cb5e9cfeea1b978de14f660ddc1"
server_key_iv: "ecb272948e3e6ecf15dd4bd590a499b8"
server_key_version: "encryptionkey@0"
I20260619 02:48:59.063081 5142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:59.084519 5142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:59.084829 5142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:59.084951 5142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:59.085155 5142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:59.085461 5142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:59.085493 5142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:59.085520 5142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:59.085544 5142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:59.090973 5142 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:40977
I20260619 02:48:59.091028 5269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:40977 every 8 connection(s)
I20260619 02:48:59.091341 5142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:48:59.096174 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.091972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:58359 (local address 127.19.113.126:34293)
0619 02:48:59.092035 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.092036 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.092261 (+ 225us) server_negotiation.cc:407] Connection header received
0619 02:48:59.092716 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.092717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.092727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.092787 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:59.093500 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.094003 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.094981 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.095190 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.095249 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.095960 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:59.095961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:59.095961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:48:59.095965 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:59.095979 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:48:59.095983 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:59.096026 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.096086 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.096114 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:48:59.096354 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.091965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:34293 (local address 127.19.113.66:58359)
0619 02:48:59.092138 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:59.092186 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:59.092637 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:59.092864 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:59.092868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:59.092876 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:59.093409 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.093414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.094070 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.094071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.094896 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.094899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.095267 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.095268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.095377 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.095834 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:48:59.095834 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:59.095861 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:48:59.095883 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:59.096030 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:59.096032 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:59.096034 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:48:59.096078 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:48:59.096152 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260619 02:48:59.096808 5270 heartbeater.cc:347] Connected to a master server at 127.19.113.126:34293
I20260619 02:48:59.096884 5270 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:59.097043 5270 heartbeater.cc:511] Master 127.19.113.126:34293 requested a full tablet report, sending...
I20260619 02:48:59.097373 4951 ts_manager.cc:194] Registered new tserver with Master: 8215745235bd4a36b9eb1ec12e35041a (127.19.113.66:40977)
I20260619 02:48:59.097728 4951 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.66:58359
I20260619 02:48:59.099040 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 5142
I20260619 02:48:59.099123 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:48:59.099308 19909 external_mini_cluster.cc:1428] Setting key 2e39369fc3e5c48b9352f43edc4af7eb
I20260619 02:48:59.100078 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:34293
--builtin_ntp_servers=127.19.113.84:35579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:48:59.177135 5273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:48:59.177338 5273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:48:59.177366 5273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:48:59.178484 5273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:48:59.178524 5273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:48:59.178555 5273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:48:59.179880 5273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:34293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:48:59.180107 5273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:48:59.180365 5273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:48:59.185009 5273 server_base.cc:1034] running on GCE node
W20260619 02:48:59.184937 5281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:59.184947 5279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:48:59.184937 5278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:48:59.185303 5273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:48:59.185539 5273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:48:59.186688 5273 hybrid_clock.cc:648] HybridClock initialized: now 1781837339186675 us; error 26 us; skew 500 ppm
I20260619 02:48:59.187880 5273 webserver.cc:460] Webserver started at http://127.19.113.67:43267/ using document root <none> and password file <none>
I20260619 02:48:59.188107 5273 fs_manager.cc:359] Metadata directory not provided
I20260619 02:48:59.188154 5273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:48:59.188279 5273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:48:59.189333 5273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "bb16e93036d6492ba8b927390863691b"
format_stamp: "Formatted at 2026-06-19 02:48:59 on dist-test-slave-zqpb"
server_key: "f160ea29133f8fbcc7d2a18905d10b42"
server_key_iv: "4eef351581d1b52b8a74ddb2f3723654"
server_key_version: "encryptionkey@0"
I20260619 02:48:59.189659 5273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "bb16e93036d6492ba8b927390863691b"
format_stamp: "Formatted at 2026-06-19 02:48:59 on dist-test-slave-zqpb"
server_key: "f160ea29133f8fbcc7d2a18905d10b42"
server_key_iv: "4eef351581d1b52b8a74ddb2f3723654"
server_key_version: "encryptionkey@0"
I20260619 02:48:59.190893 5273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:48:59.191565 5287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:59.191720 5273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:48:59.191797 5273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "bb16e93036d6492ba8b927390863691b"
format_stamp: "Formatted at 2026-06-19 02:48:59 on dist-test-slave-zqpb"
server_key: "f160ea29133f8fbcc7d2a18905d10b42"
server_key_iv: "4eef351581d1b52b8a74ddb2f3723654"
server_key_version: "encryptionkey@0"
I20260619 02:48:59.191864 5273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:48:59.200362 5273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:48:59.200605 5273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:48:59.200726 5273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:48:59.200904 5273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:48:59.201198 5273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:48:59.201233 5273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:59.201260 5273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:48:59.201283 5273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:48:59.206581 5273 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:35315
I20260619 02:48:59.206631 5400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:35315 every 8 connection(s)
I20260619 02:48:59.206997 5273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:48:59.211494 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.207615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:50905 (local address 127.19.113.126:34293)
0619 02:48:59.207695 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.207697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.207848 (+ 151us) server_negotiation.cc:407] Connection header received
0619 02:48:59.208308 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.208309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.208319 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.208366 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:59.208961 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.209453 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.210348 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.210528 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.210577 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.211304 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:59.211305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:59.211305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:48:59.211309 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:59.211321 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:48:59.211325 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:59.211363 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.211412 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.211436 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:48:59.211664 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.207586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:34293 (local address 127.19.113.67:50905)
0619 02:48:59.207742 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:59.207774 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:59.208235 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:59.208418 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:59.208421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:59.208429 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:48:59.208901 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.208904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.209511 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.209512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.210283 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.210285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.210584 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.210585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.210698 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.211189 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:48:59.211189 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:48:59.211216 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:48:59.211238 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:48:59.211364 (+ 126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:48:59.211366 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:48:59.211368 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:48:59.211410 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:48:59.211483 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260619 02:48:59.212095 5401 heartbeater.cc:347] Connected to a master server at 127.19.113.126:34293
I20260619 02:48:59.212179 5401 heartbeater.cc:464] Registering TS with master...
I20260619 02:48:59.212342 5401 heartbeater.cc:511] Master 127.19.113.126:34293 requested a full tablet report, sending...
I20260619 02:48:59.212684 4951 ts_manager.cc:194] Registered new tserver with Master: bb16e93036d6492ba8b927390863691b (127.19.113.67:35315)
I20260619 02:48:59.213017 4951 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.67:50905
I20260619 02:48:59.214133 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 5273
I20260619 02:48:59.214233 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:48:59.214399 19909 external_mini_cluster.cc:1428] Setting key db4ac0033915a596edf88ba32ffb2168
I20260619 02:48:59.215281 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:48:59.219233 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.216556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.19.113.126:34293)
0619 02:48:59.216604 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.216605 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.216754 (+ 149us) server_negotiation.cc:407] Connection header received
0619 02:48:59.216780 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.216781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.216792 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.216837 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:48:59.216959 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.217496 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.218251 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.218433 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.218480 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.219040 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:48:59.219041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:48:59.219042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:48:59.219045 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:48:59.219058 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:48:59.219062 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:48:59.219101 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.219135 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.219162 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:48:59.220558 4951 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:48:59.222044 4951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:48:59.227988 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.224089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.19.113.65:33655)
0619 02:48:59.224264 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.224268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.224354 (+ 86us) server_negotiation.cc:407] Connection header received
0619 02:48:59.224469 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.224472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.224486 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.224552 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:59.224773 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.225518 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.226794 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.227564 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.227634 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.227722 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.227858 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.227903 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
I20260619 02:48:59.228282 5409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.223970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:33655 (local address 127.0.0.1:48258)
0619 02:48:59.224291 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:59.224299 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:59.224379 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:59.224562 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:59.224565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:59.224568 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:59.224627 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.224628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.225595 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.225595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.226696 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.226699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.227656 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.227657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.227738 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.227752 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:48:59.228137 (+ 385us) client_negotiation.cc:240] Negotiation successful
0619 02:48:59.228211 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260619 02:48:59.228579 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.224145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:35315 (local address 127.0.0.1:54468)
0619 02:48:59.224324 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:59.224334 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:59.224385 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:59.224628 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:59.224629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:59.224630 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:59.224659 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.224678 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.225965 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.225966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.227045 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.227048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.228404 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.228405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.228470 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.228475 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:48:59.228499 (+ 24us) client_negotiation.cc:240] Negotiation successful
0619 02:48:59.228529 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260619 02:48:59.228976 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.224263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.19.113.67:35315)
0619 02:48:59.224480 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.224483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.224491 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:48:59.224521 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.224524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.224536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.224626 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:59.224791 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.225884 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.227752 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.228305 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.228748 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.228840 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.228863 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.228899 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":63,"threads_started":1}
I20260619 02:48:59.229039 5073 tablet_service.cc:1467] Processing CreateTablet for tablet 4e695342dd7a436393a39ff79a78c0b3 (DEFAULT_TABLE table=test-table [id=9064ab0f74b1470bb39615ab838902e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:59.229358 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.223948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:40977 (local address 127.0.0.1:37148)
0619 02:48:59.224225 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:59.224266 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:59.224331 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:59.224640 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:59.224641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:59.224643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:59.224687 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.224689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.227180 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.227181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.228182 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.228185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.229146 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.229147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.229206 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.229209 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:59.229257 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:48:59.229297 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":62,"threads_started":1}
I20260619 02:48:59.229398 5411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.224045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.19.113.66:40977)
0619 02:48:59.224431 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.224435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.224446 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:48:59.224484 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.224486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.224501 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.224572 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:59.224759 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.225879 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.228269 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.228941 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.229006 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.229089 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.229286 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.229324 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":81,"threads_started":1}
I20260619 02:48:59.229398 5073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e695342dd7a436393a39ff79a78c0b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:59.230000 5204 tablet_service.cc:1467] Processing CreateTablet for tablet 4e695342dd7a436393a39ff79a78c0b3 (DEFAULT_TABLE table=test-table [id=9064ab0f74b1470bb39615ab838902e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:59.230329 5204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e695342dd7a436393a39ff79a78c0b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:59.231355 5335 tablet_service.cc:1467] Processing CreateTablet for tablet 4e695342dd7a436393a39ff79a78c0b3 (DEFAULT_TABLE table=test-table [id=9064ab0f74b1470bb39615ab838902e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:48:59.231622 5335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e695342dd7a436393a39ff79a78c0b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:48:59.232755 5415 tablet_bootstrap.cc:492] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f: Bootstrap starting.
I20260619 02:48:59.233441 5416 tablet_bootstrap.cc:492] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Bootstrap starting.
I20260619 02:48:59.233472 5415 tablet_bootstrap.cc:654] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:59.233851 5415 log.cc:824] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:59.233959 5416 tablet_bootstrap.cc:654] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:59.234278 5416 log.cc:824] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:59.234356 5418 tablet_bootstrap.cc:492] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Bootstrap starting.
I20260619 02:48:59.234704 5415 tablet_bootstrap.cc:492] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f: No bootstrap required, opened a new log
I20260619 02:48:59.234774 5415 ts_tablet_manager.cc:1397] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:48:59.234952 5418 tablet_bootstrap.cc:654] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Neither blocks nor log segments found. Creating new log.
I20260619 02:48:59.235205 5418 log.cc:824] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:48:59.235383 5416 tablet_bootstrap.cc:492] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: No bootstrap required, opened a new log
I20260619 02:48:59.235453 5416 ts_tablet_manager.cc:1397] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:48:59.235783 5418 tablet_bootstrap.cc:492] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: No bootstrap required, opened a new log
I20260619 02:48:59.235839 5418 ts_tablet_manager.cc:1397] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:48:59.237221 5415 raft_consensus.cc:348] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.237388 5415 raft_consensus.cc:374] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:59.237422 5415 raft_consensus.cc:729] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5754f51f350d4452b64919a8f8b9850f, State: Initialized, Role: FOLLOWER
I20260619 02:48:59.237560 5415 consensus_queue.cc:260] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.237587 5418 raft_consensus.cc:348] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.237723 5418 raft_consensus.cc:374] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:59.237763 5418 raft_consensus.cc:729] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb16e93036d6492ba8b927390863691b, State: Initialized, Role: FOLLOWER
I20260619 02:48:59.237812 5415 ts_tablet_manager.cc:1428] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:48:59.237938 5139 heartbeater.cc:503] Master 127.19.113.126:34293 was elected leader, sending a full tablet report...
I20260619 02:48:59.237818 5416 raft_consensus.cc:348] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.238008 5416 raft_consensus.cc:374] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:48:59.238034 5416 raft_consensus.cc:729] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8215745235bd4a36b9eb1ec12e35041a, State: Initialized, Role: FOLLOWER
I20260619 02:48:59.238113 5416 consensus_queue.cc:260] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.237871 5418 consensus_queue.cc:260] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.238329 5416 ts_tablet_manager.cc:1428] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:48:59.238395 5418 ts_tablet_manager.cc:1428] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 02:48:59.238500 5401 heartbeater.cc:503] Master 127.19.113.126:34293 was elected leader, sending a full tablet report...
I20260619 02:48:59.238579 5270 heartbeater.cc:503] Master 127.19.113.126:34293 was elected leader, sending a full tablet report...
W20260619 02:48:59.341830 5271 tablet.cc:2369] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:48:59.457487 5402 tablet.cc:2369] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:48:59.460369 5140 tablet.cc:2369] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:48:59.532989 5422 raft_consensus.cc:482] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:48:59.533110 5422 raft_consensus.cc:504] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.533480 5422 leader_election.cc:288] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb16e93036d6492ba8b927390863691b (127.19.113.67:35315), 5754f51f350d4452b64919a8f8b9850f (127.19.113.65:33655)
I20260619 02:48:59.536089 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.533755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:40109 (local address 127.19.113.65:33655)
0619 02:48:59.533803 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.533805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.533813 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:48:59.533843 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.533843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.533853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.533899 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:59.534041 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.534526 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.535465 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.535807 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.535866 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.535875 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.535989 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.536022 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:48:59.536089 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.533642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:33655 (local address 127.19.113.66:40109)
0619 02:48:59.533726 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:59.533739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:59.533788 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:59.533907 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:59.533908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:59.533909 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:59.533979 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.533981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.534594 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.534595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.535384 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.535387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.535868 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.535869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.535929 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.535932 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:59.535975 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:48:59.536016 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 02:48:59.536250 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.533712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:49181 (local address 127.19.113.67:35315)
0619 02:48:59.533762 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.533764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.533969 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:48:59.534023 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.534025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.534034 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.534077 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:48:59.534217 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.534661 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.535589 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.535938 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.535997 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.536005 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.536159 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.536192 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:48:59.536254 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.533643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:35315 (local address 127.19.113.66:49181)
0619 02:48:59.533901 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:59.533910 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:48:59.533965 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:48:59.534088 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:48:59.534089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:48:59.534090 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:48:59.534157 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.534159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.534729 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.534730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.535505 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:48:59.535507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.536009 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.536010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:48:59.536090 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.536093 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:48:59.536136 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:48:59.536175 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":103,"thread_start_us":92,"threads_started":1}
I20260619 02:48:59.536409 5093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e695342dd7a436393a39ff79a78c0b3" candidate_uuid: "8215745235bd4a36b9eb1ec12e35041a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5754f51f350d4452b64919a8f8b9850f" is_pre_election: true
I20260619 02:48:59.536569 5093 raft_consensus.cc:2454] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8215745235bd4a36b9eb1ec12e35041a in term 0.
I20260619 02:48:59.536542 5355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e695342dd7a436393a39ff79a78c0b3" candidate_uuid: "8215745235bd4a36b9eb1ec12e35041a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb16e93036d6492ba8b927390863691b" is_pre_election: true
I20260619 02:48:59.536657 5355 raft_consensus.cc:2454] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8215745235bd4a36b9eb1ec12e35041a in term 0.
I20260619 02:48:59.536805 5158 leader_election.cc:302] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5754f51f350d4452b64919a8f8b9850f, 8215745235bd4a36b9eb1ec12e35041a; no voters:
I20260619 02:48:59.536970 5422 raft_consensus.cc:2781] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:48:59.537020 5422 raft_consensus.cc:482] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:48:59.537034 5422 raft_consensus.cc:3037] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:59.537698 5422 raft_consensus.cc:504] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.537813 5422 leader_election.cc:288] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [CANDIDATE]: Term 1 election: Requested vote from peers bb16e93036d6492ba8b927390863691b (127.19.113.67:35315), 5754f51f350d4452b64919a8f8b9850f (127.19.113.65:33655)
I20260619 02:48:59.537949 5355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e695342dd7a436393a39ff79a78c0b3" candidate_uuid: "8215745235bd4a36b9eb1ec12e35041a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb16e93036d6492ba8b927390863691b"
I20260619 02:48:59.537966 5093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e695342dd7a436393a39ff79a78c0b3" candidate_uuid: "8215745235bd4a36b9eb1ec12e35041a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5754f51f350d4452b64919a8f8b9850f"
I20260619 02:48:59.538025 5093 raft_consensus.cc:3037] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:59.538028 5355 raft_consensus.cc:3037] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:48:59.538707 5093 raft_consensus.cc:2454] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8215745235bd4a36b9eb1ec12e35041a in term 1.
I20260619 02:48:59.538774 5355 raft_consensus.cc:2454] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8215745235bd4a36b9eb1ec12e35041a in term 1.
I20260619 02:48:59.538861 5158 leader_election.cc:302] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5754f51f350d4452b64919a8f8b9850f, 8215745235bd4a36b9eb1ec12e35041a; no voters:
I20260619 02:48:59.538987 5422 raft_consensus.cc:2781] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:48:59.539132 5422 raft_consensus.cc:686] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 1 LEADER]: Becoming Leader. State: Replica: 8215745235bd4a36b9eb1ec12e35041a, State: Running, Role: LEADER
I20260619 02:48:59.539222 5422 consensus_queue.cc:237] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } }
I20260619 02:48:59.539990 4950 catalog_manager.cc:5495] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8215745235bd4a36b9eb1ec12e35041a (127.19.113.66). New cstate: current_term: 1 leader_uuid: "8215745235bd4a36b9eb1ec12e35041a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8215745235bd4a36b9eb1ec12e35041a" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 40977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 } health_report { overall_health: UNKNOWN } } }
I20260619 02:48:59.580966 5411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.578816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.19.113.66:40977)
0619 02:48:59.578858 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:48:59.578860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:48:59.578891 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:48:59.578963 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:48:59.578964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:48:59.578986 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:48:59.579052 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:48:59.579155 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.579625 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.580296 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:48:59.580478 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:48:59.580523 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:48:59.580639 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:48:59.580733 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:48:59.580788 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:48:59.580867 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 02:48:59.580881 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:48:59.582616 5093 raft_consensus.cc:1261] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 1 FOLLOWER]: Refusing update from remote peer 8215745235bd4a36b9eb1ec12e35041a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:48:59.582610 5355 raft_consensus.cc:1261] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 1 FOLLOWER]: Refusing update from remote peer 8215745235bd4a36b9eb1ec12e35041a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:48:59.582948 5425 consensus_queue.cc:1035] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5754f51f350d4452b64919a8f8b9850f" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 33655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:48:59.583081 5422 consensus_queue.cc:1035] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb16e93036d6492ba8b927390863691b" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 35315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:48:59.585532 5429 mvcc.cc:205] Tried to move back new op lower bound from 7298405742928048128 to 7298405742753116160. Current Snapshot: MvccSnapshot[applied={T|T < 7298405742928048128}]
I20260619 02:48:59.587370 4949 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:48:59.587455 4949 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:48:59.588181 4949 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc5613779ada4f6ea32cc7ed2410d68e: Sending DeleteTablet for 3 replicas of tablet 4e695342dd7a436393a39ff79a78c0b3
I20260619 02:48:59.588488 5073 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e695342dd7a436393a39ff79a78c0b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:59 UTC) from {username='slave'} at 127.0.0.1:48258
I20260619 02:48:59.588547 5204 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e695342dd7a436393a39ff79a78c0b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:59 UTC) from {username='slave'} at 127.0.0.1:37148
I20260619 02:48:59.588757 5438 tablet_replica.cc:330] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f: stopping tablet replica
I20260619 02:48:59.588775 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e695342dd7a436393a39ff79a78c0b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:48:59 UTC) from {username='slave'} at 127.0.0.1:54468
I20260619 02:48:59.588851 5438 raft_consensus.cc:2229] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:48:59.588956 5438 raft_consensus.cc:2258] T 4e695342dd7a436393a39ff79a78c0b3 P 5754f51f350d4452b64919a8f8b9850f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:59.588963 5439 tablet_replica.cc:330] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: stopping tablet replica
I20260619 02:48:59.589001 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 5011
I20260619 02:48:59.589036 5439 raft_consensus.cc:2229] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:48:59.589108 5439 raft_consensus.cc:2258] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:59.589221 5440 tablet_replica.cc:330] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: stopping tablet replica
I20260619 02:48:59.589284 5440 raft_consensus.cc:2229] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:48:59.589349 5440 raft_consensus.cc:2258] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:48:59.589790 5440 ts_tablet_manager.cc:1905] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:59.591521 5439 ts_tablet_manager.cc:1905] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:48:59.592628 5440 ts_tablet_manager.cc:1918] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:59.592684 5440 log.cc:1196] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/4e695342dd7a436393a39ff79a78c0b3
I20260619 02:48:59.592849 5440 ts_tablet_manager.cc:1939] T 4e695342dd7a436393a39ff79a78c0b3 P bb16e93036d6492ba8b927390863691b: Deleting consensus metadata
I20260619 02:48:59.593176 4935 catalog_manager.cc:4841] TS bb16e93036d6492ba8b927390863691b (127.19.113.67:35315): tablet 4e695342dd7a436393a39ff79a78c0b3 (table test-table [id=9064ab0f74b1470bb39615ab838902e4]) successfully deleted
I20260619 02:48:59.595147 5439 ts_tablet_manager.cc:1918] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:48:59.595216 5439 log.cc:1196] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/4e695342dd7a436393a39ff79a78c0b3
I20260619 02:48:59.595428 5439 ts_tablet_manager.cc:1939] T 4e695342dd7a436393a39ff79a78c0b3 P 8215745235bd4a36b9eb1ec12e35041a: Deleting consensus metadata
I20260619 02:48:59.595795 4935 catalog_manager.cc:4841] TS 8215745235bd4a36b9eb1ec12e35041a (127.19.113.66:40977): tablet 4e695342dd7a436393a39ff79a78c0b3 (table test-table [id=9064ab0f74b1470bb39615ab838902e4]) successfully deleted
W20260619 02:48:59.596321 4936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:33655 (error 108)
I20260619 02:48:59.596524 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 5142
I20260619 02:48:59.596733 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:48:59.596627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:33655 (local address 127.0.0.1:48266)
0619 02:48:59.596671 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:48:59.596704 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:33655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 02:48:59.596823 4936 catalog_manager.cc:4584] TS 5754f51f350d4452b64919a8f8b9850f (127.19.113.65:33655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e695342dd7a436393a39ff79a78c0b3: Network error: Client connection negotiation failed: client connection to 127.19.113.65:33655: connect: Connection refused (error 111)
I20260619 02:48:59.601438 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 5273
I20260619 02:48:59.605139 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 4920
2026-06-19T02:48:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (932 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:48:59 dist-test-slave-zqpb krb5kdc[5444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:48:59 dist-test-slave-zqpb krb5kdc[5444](info): set up 2 sockets
Jun 19 02:48:59 dist-test-slave-zqpb krb5kdc[5444](info): commencing operation
krb5kdc: starting...
W20260619 02:49:00.562428 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:00 dist-test-slave-zqpb krb5kdc[5444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:00.674590 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40627
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:38607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:40627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:00.754413 5457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:00.754622 5457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:00.754644 5457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:00.755750 5457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:00.755784 5457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:00.755797 5457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:00.755808 5457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:00.755818 5457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:00.757218 5457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:38607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:40627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:40627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:00.757402 5457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:00.757568 5457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:00.762084 5465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:00.762084 5462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:00.762115 5463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:00.762208 5457 server_base.cc:1034] running on GCE node
I20260619 02:49:00.762506 5457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:00.762777 5457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:00.763928 5457 hybrid_clock.cc:648] HybridClock initialized: now 1781837340763921 us; error 37 us; skew 500 ppm
Jun 19 02:49:00 dist-test-slave-zqpb krb5kdc[5444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837340, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:00.766460 5457 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:00.766911 5457 webserver.cc:460] Webserver started at http://127.19.113.126:41451/ using document root <none> and password file <none>
I20260619 02:49:00.767109 5457 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:00.767165 5457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:00.767292 5457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:00.768338 5457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "4543dcb0138c433e8baf2289f1bbb264"
format_stamp: "Formatted at 2026-06-19 02:49:00 on dist-test-slave-zqpb"
server_key: "3621490b6b87fd8061c4f89fd7b90cdf"
server_key_iv: "10cd14bea2d9907c594153422673de11"
server_key_version: "encryptionkey@0"
I20260619 02:49:00.768666 5457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "4543dcb0138c433e8baf2289f1bbb264"
format_stamp: "Formatted at 2026-06-19 02:49:00 on dist-test-slave-zqpb"
server_key: "3621490b6b87fd8061c4f89fd7b90cdf"
server_key_iv: "10cd14bea2d9907c594153422673de11"
server_key_version: "encryptionkey@0"
I20260619 02:49:00.769903 5457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:00.770664 5472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:00.770833 5457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:00.770931 5457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "4543dcb0138c433e8baf2289f1bbb264"
format_stamp: "Formatted at 2026-06-19 02:49:00 on dist-test-slave-zqpb"
server_key: "3621490b6b87fd8061c4f89fd7b90cdf"
server_key_iv: "10cd14bea2d9907c594153422673de11"
server_key_version: "encryptionkey@0"
I20260619 02:49:00.770996 5457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:00.783174 5457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:00.784963 5457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:00.785090 5457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:00.789093 5457 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:40627
I20260619 02:49:00.789139 5524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:40627 every 8 connection(s)
I20260619 02:49:00.789448 5457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:00.790273 5525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:00.792477 5525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264: Bootstrap starting.
I20260619 02:49:00.792979 5525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:00.793202 5525 log.cc:824] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:00.793776 5525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264: No bootstrap required, opened a new log
I20260619 02:49:00.795347 5525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4543dcb0138c433e8baf2289f1bbb264" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40627 } }
I20260619 02:49:00.795476 5525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:00.795496 5525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4543dcb0138c433e8baf2289f1bbb264, State: Initialized, Role: FOLLOWER
I20260619 02:49:00.795603 5525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4543dcb0138c433e8baf2289f1bbb264" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40627 } }
I20260619 02:49:00.795676 5525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:00.795704 5525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:00.795737 5525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:00.796198 5525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4543dcb0138c433e8baf2289f1bbb264" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40627 } }
I20260619 02:49:00.796274 5525 leader_election.cc:302] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4543dcb0138c433e8baf2289f1bbb264; no voters:
I20260619 02:49:00.796388 5525 leader_election.cc:288] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:00.796430 5528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:00.796583 5528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [term 1 LEADER]: Becoming Leader. State: Replica: 4543dcb0138c433e8baf2289f1bbb264, State: Running, Role: LEADER
I20260619 02:49:00.796593 5525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:00.796676 5528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4543dcb0138c433e8baf2289f1bbb264" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40627 } }
I20260619 02:49:00.797010 5530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4543dcb0138c433e8baf2289f1bbb264. Latest consensus state: current_term: 1 leader_uuid: "4543dcb0138c433e8baf2289f1bbb264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4543dcb0138c433e8baf2289f1bbb264" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40627 } } }
I20260619 02:49:00.797086 5530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:00.797132 5529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4543dcb0138c433e8baf2289f1bbb264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4543dcb0138c433e8baf2289f1bbb264" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 40627 } } }
I20260619 02:49:00.797192 5529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:00.797914 5536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:00.798384 5536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:00.799325 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 5457
I20260619 02:49:00.799437 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:00.799624 19909 external_mini_cluster.cc:1428] Setting key 1c0b632141add7aa4beed2b5fd9326f5
I20260619 02:49:00.799958 5536 catalog_manager.cc:1347] Generated new cluster ID: ffaa2876005f4763868eb941671c501f
I20260619 02:49:00.800006 5536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:00.807262 5536 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:00.807844 5536 catalog_manager.cc:1504] Loading token signing keys...
Jun 19 02:49:00 dist-test-slave-zqpb krb5kdc[5444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:00.812659 5536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264: Generated new TSK 0
I20260619 02:49:00.812808 5536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:00.813673 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:00.802976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.19.113.126:40627)
0619 02:49:00.803157 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:00.803166 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:00.803201 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:49:00.803315 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:00.803323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:00.803388 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:00.803528 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:00.804237 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:00.806156 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:00.806925 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:00.807514 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:00.807613 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:00.810292 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:00.810296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:00.810296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:00.810308 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:00.812148 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:00.812519 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:00.812522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:00.812522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:00.812638 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:00.812869 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:00.812870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:00.812870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:00.813102 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:00.813211 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:00.813345 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 02:49:00.813453 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":183,"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:49:00.837179 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40627
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:38607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:00.916460 5551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:00.916666 5551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:00.916698 5551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:00.917824 5551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:00.917866 5551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:00.917900 5551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:49:00.919365 5551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:38607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:00.919585 5551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:00.919795 5551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:00.924467 5556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:00.924515 5559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:00.924592 5551 server_base.cc:1034] running on GCE node
W20260619 02:49:00.924517 5557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:00.924851 5551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:00.925047 5551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:00.926258 5551 hybrid_clock.cc:648] HybridClock initialized: now 1781837340926239 us; error 31 us; skew 500 ppm
Jun 19 02:49:00 dist-test-slave-zqpb krb5kdc[5444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837340, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:00.928794 5551 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:49:00.929260 5551 webserver.cc:460] Webserver started at http://127.19.113.65:36905/ using document root <none> and password file <none>
I20260619 02:49:00.929462 5551 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:00.929503 5551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:00.929617 5551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:00.930684 5551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "730c0382c20d4ffc8e2f8c9f91c32960"
format_stamp: "Formatted at 2026-06-19 02:49:00 on dist-test-slave-zqpb"
server_key: "a8c24a6552677c821a131192fe81911c"
server_key_iv: "6b78b74113c96f385e547d0af2d4331e"
server_key_version: "encryptionkey@0"
I20260619 02:49:00.931030 5551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "730c0382c20d4ffc8e2f8c9f91c32960"
format_stamp: "Formatted at 2026-06-19 02:49:00 on dist-test-slave-zqpb"
server_key: "a8c24a6552677c821a131192fe81911c"
server_key_iv: "6b78b74113c96f385e547d0af2d4331e"
server_key_version: "encryptionkey@0"
I20260619 02:49:00.932288 5551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:49:00.933002 5566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:00.933185 5551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:49:00.933264 5551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "730c0382c20d4ffc8e2f8c9f91c32960"
format_stamp: "Formatted at 2026-06-19 02:49:00 on dist-test-slave-zqpb"
server_key: "a8c24a6552677c821a131192fe81911c"
server_key_iv: "6b78b74113c96f385e547d0af2d4331e"
server_key_version: "encryptionkey@0"
I20260619 02:49:00.933314 5551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:00.945832 5551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:00.947614 5551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:00.947727 5551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:00.947922 5551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:00.948189 5551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:00.948220 5551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:00.948238 5551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:00.948254 5551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:00.953497 5551 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:46253
I20260619 02:49:00.953567 5679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:46253 every 8 connection(s)
I20260619 02:49:00.953827 5551 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:49:00 dist-test-slave-zqpb krb5kdc[5444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837340, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:00.962005 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 5551
I20260619 02:49:00.962100 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:49:00.962321 19909 external_mini_cluster.cc:1428] Setting key 82e8604f784d56a830393bb8d4abbb36
I20260619 02:49:00.962450 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:00.954478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:41067 (local address 127.19.113.126:40627)
0619 02:49:00.954531 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:00.954532 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:00.954730 (+ 198us) server_negotiation.cc:407] Connection header received
0619 02:49:00.955272 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:00.955273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:00.955286 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:00.955336 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:00.956256 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:00.956754 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:00.957691 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:00.957895 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:00.957947 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:00.960015 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:00.960017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:00.960017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:00.960024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:00.961120 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:00.961483 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:00.961484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:00.961484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:00.961539 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:00.961768 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:00.961769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:00.961770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:00.961891 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:00.961960 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:00.962267 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 02:49:00.962363 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:49:00.962612 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:00.954454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40627 (local address 127.19.113.65:41067)
0619 02:49:00.954611 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:00.954653 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:00.955181 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:00.955402 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:00.955406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:00.955961 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:00.956151 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:00.956157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:00.956831 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:00.956832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:00.957612 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:00.957614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:00.957961 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:00.957962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:00.958087 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:00.958532 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:00.958540 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:00.959923 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:00.961206 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:00.961213 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:00.961215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:00.961417 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:00.961591 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:00.961592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:00.961592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:00.961709 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:00.961971 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:00.961978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:00.962099 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 02:49:00.962215 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:49:00.962369 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260619 02:49:00.963126 5680 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40627
I20260619 02:49:00.963220 5680 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:00.963385 5680 heartbeater.cc:511] Master 127.19.113.126:40627 requested a full tablet report, sending...
I20260619 02:49:00.963908 5489 ts_manager.cc:194] Registered new tserver with Master: 730c0382c20d4ffc8e2f8c9f91c32960 (127.19.113.65:46253)
I20260619 02:49:00.964648 5489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:41067
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:49:00.985740 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40627
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:38607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:01.064389 5685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:01.064584 5685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:01.064607 5685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:01.065682 5685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:01.065717 5685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:01.065742 5685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:49:01.067147 5685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:38607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:01.067339 5685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:01.067507 5685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:01.072288 5691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.072285 5690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:01.072364 5685 server_base.cc:1034] running on GCE node
W20260619 02:49:01.072286 5693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:01.072696 5685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:01.072899 5685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:01.074041 5685 hybrid_clock.cc:648] HybridClock initialized: now 1781837341074027 us; error 32 us; skew 500 ppm
Jun 19 02:49:01 dist-test-slave-zqpb krb5kdc[5444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:01.076444 5685 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:49:01.076831 5685 webserver.cc:460] Webserver started at http://127.19.113.66:37949/ using document root <none> and password file <none>
I20260619 02:49:01.076999 5685 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:01.077035 5685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:01.077126 5685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:01.078148 5685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "7f27d3989baf44adab4cf63d16316b5f"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "21a0be9d48144c1337154fe0b9dab4db"
server_key_iv: "913f9693bc7cd0c2bb32a1775b7d0b26"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.078436 5685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "7f27d3989baf44adab4cf63d16316b5f"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "21a0be9d48144c1337154fe0b9dab4db"
server_key_iv: "913f9693bc7cd0c2bb32a1775b7d0b26"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.079659 5685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:01.080286 5700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.080430 5685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:01.080502 5685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "7f27d3989baf44adab4cf63d16316b5f"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "21a0be9d48144c1337154fe0b9dab4db"
server_key_iv: "913f9693bc7cd0c2bb32a1775b7d0b26"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.080572 5685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:01.096561 5685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:01.098711 5685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:01.098835 5685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:01.099097 5685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:01.099387 5685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:01.099421 5685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.099458 5685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:01.099481 5685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.104868 5685 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:46459
I20260619 02:49:01.104910 5813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:46459 every 8 connection(s)
I20260619 02:49:01.105204 5685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:49:01.110319 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 5685
I20260619 02:49:01.110426 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:49:01.110627 19909 external_mini_cluster.cc:1428] Setting key 0b8a94b7623e66391d3f65ca93f09ef1
Jun 19 02:49:01 dist-test-slave-zqpb krb5kdc[5444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:01.114571 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.105853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:48057 (local address 127.19.113.126:40627)
0619 02:49:01.105915 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.105917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.106137 (+ 220us) server_negotiation.cc:407] Connection header received
0619 02:49:01.106601 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.106602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.106611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.106666 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:01.107624 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.108147 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.109171 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.109368 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.109423 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.111827 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:01.111828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:01.111829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:01.111835 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:01.113291 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:01.113695 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:01.113696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:01.113696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:01.113759 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:01.113959 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:01.113960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:01.113961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:01.114098 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:01.114161 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.114382 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.114471 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:49:01.114662 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.105822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40627 (local address 127.19.113.66:48057)
0619 02:49:01.105976 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.106017 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.106516 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.106719 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.106722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.107311 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:01.107507 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.107512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.108241 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.108241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.109088 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.109090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.109457 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.109458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.109584 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.110043 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:01.110053 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:01.111737 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:01.113378 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:01.113384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:01.113386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:01.113616 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:01.113813 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:01.113813 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:01.113813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:01.113910 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:01.114152 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:01.114156 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:01.114269 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.114351 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.114476 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260619 02:49:01.115154 5814 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40627
I20260619 02:49:01.115264 5814 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:01.115456 5814 heartbeater.cc:511] Master 127.19.113.126:40627 requested a full tablet report, sending...
I20260619 02:49:01.115906 5489 ts_manager.cc:194] Registered new tserver with Master: 7f27d3989baf44adab4cf63d16316b5f (127.19.113.66:46459)
I20260619 02:49:01.116334 5489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:48057
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:49:01.133958 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40627
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:38607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:01.212249 5819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:01.212440 5819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:01.212461 5819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:01.213579 5819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:01.213617 5819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:01.213644 5819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:49:01.215065 5819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:38607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:01.215281 5819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:01.215479 5819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:49:01.220208 5819 server_base.cc:1034] running on GCE node
W20260619 02:49:01.220201 5824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.220176 5825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.220192 5827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:01.220613 5819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:01.220845 5819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:01.221979 5819 hybrid_clock.cc:648] HybridClock initialized: now 1781837341221969 us; error 28 us; skew 500 ppm
Jun 19 02:49:01 dist-test-slave-zqpb krb5kdc[5444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:01.224419 5819 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:49:01.224852 5819 webserver.cc:460] Webserver started at http://127.19.113.67:39551/ using document root <none> and password file <none>
I20260619 02:49:01.225045 5819 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:01.225093 5819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:01.225221 5819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:01.226219 5819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "b46e56f9a73342729b80b0ca2185a1c6"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "92f9d27e7835215a4fd31e75a9352e86"
server_key_iv: "219de5d38066ecbb4379bdcc224baa4c"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.226536 5819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "b46e56f9a73342729b80b0ca2185a1c6"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "92f9d27e7835215a4fd31e75a9352e86"
server_key_iv: "219de5d38066ecbb4379bdcc224baa4c"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.227876 5819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:01.228664 5834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.228847 5819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:49:01.228935 5819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "b46e56f9a73342729b80b0ca2185a1c6"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "92f9d27e7835215a4fd31e75a9352e86"
server_key_iv: "219de5d38066ecbb4379bdcc224baa4c"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.228996 5819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:01.247920 5819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:01.249778 5819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:01.249910 5819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:01.250144 5819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:01.250432 5819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:01.250464 5819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.250494 5819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:01.250511 5819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.255805 5819 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:33979
I20260619 02:49:01.255846 5947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:33979 every 8 connection(s)
I20260619 02:49:01.256138 5819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:49:01.258394 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 5819
I20260619 02:49:01.258503 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:49:01.258687 19909 external_mini_cluster.cc:1428] Setting key b8d3f854521f0b7065f9345f831f04ac
Jun 19 02:49:01 dist-test-slave-zqpb krb5kdc[5444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:01.264782 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.256760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:46423 (local address 127.19.113.126:40627)
0619 02:49:01.256824 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.256826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.257013 (+ 187us) server_negotiation.cc:407] Connection header received
0619 02:49:01.257474 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.257475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.257485 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.257541 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:01.258526 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.259244 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.260256 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.260454 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.260507 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.262442 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:01.262444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:01.262444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:01.262452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:01.263577 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:01.263938 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:01.263939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:01.263939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:01.263993 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:01.264201 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:01.264202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:01.264202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:01.264311 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:01.264366 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.264615 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.264693 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:49:01.264930 5950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.256737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:40627 (local address 127.19.113.67:46423)
0619 02:49:01.256897 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.256937 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.257400 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.257602 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.257605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.258136 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:01.258393 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.258400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.259325 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.259326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.260169 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.260172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.260520 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.260521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.260634 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.261003 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:01.261011 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:01.262348 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:01.263670 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:01.263676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:01.263678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:01.263876 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:01.264045 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:01.264046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:01.264046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:01.264148 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:01.264365 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:01.264370 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:01.264494 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.264578 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.264728 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260619 02:49:01.265317 5948 heartbeater.cc:347] Connected to a master server at 127.19.113.126:40627
I20260619 02:49:01.265405 5948 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:01.265564 5948 heartbeater.cc:511] Master 127.19.113.126:40627 requested a full tablet report, sending...
I20260619 02:49:01.265898 5489 ts_manager.cc:194] Registered new tserver with Master: b46e56f9a73342729b80b0ca2185a1c6 (127.19.113.67:33979)
I20260619 02:49:01.266253 5489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:46423
I20260619 02:49:01.270468 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:49:01.277016 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.271771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.19.113.126:40627)
0619 02:49:01.271831 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.271833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.271953 (+ 120us) server_negotiation.cc:407] Connection header received
0619 02:49:01.271994 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.271995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.272002 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.272045 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:01.272543 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.273003 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.273736 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.273911 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.273952 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.275103 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:01.275104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:01.275104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:01.275109 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:01.276095 (+ 986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:01.276427 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:01.276428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:01.276428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:01.276472 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:01.276616 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:01.276616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:01.276616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:01.276734 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:01.276804 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.276896 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.276939 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:49:01.278446 5489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:49:01.279987 5489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:49:01.285979 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.281856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46253 (local address 127.0.0.1:59374)
0619 02:49:01.281996 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.282005 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.282042 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.282595 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.282596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.282597 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.282657 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.282659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.283741 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.283742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.285104 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.285106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.285741 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.285743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.285804 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.285808 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.285854 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.285898 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":28,"spinlock_wait_cycles":768,"thread_start_us":106,"threads_started":1}
I20260619 02:49:01.286062 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.282064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.19.113.66:46459)
0619 02:49:01.282223 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.282228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.282487 (+ 259us) server_negotiation.cc:407] Connection header received
0619 02:49:01.282556 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.282558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.282573 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.282642 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.282829 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.283885 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.284852 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.285493 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.285557 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.285609 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.285943 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.285986 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260619 02:49:01.286098 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.281824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:33979 (local address 127.0.0.1:54204)
0619 02:49:01.281919 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.281955 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.282018 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.282400 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.282404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.282406 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.282474 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.282476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.283793 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.283794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.285083 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.285087 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.285919 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.285920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.285974 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.285977 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.286002 (+ 25us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.286035 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":51,"thread_start_us":52,"threads_started":1}
I20260619 02:49:01.286206 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.281856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:46459 (local address 127.0.0.1:40164)
0619 02:49:01.282411 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.282429 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.282492 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.282718 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.282719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.282721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.282757 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.282759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.283961 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.283962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.284768 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.284771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.285655 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.285656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.285732 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.285745 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.285805 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.285855 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":3,"thread_start_us":53,"threads_started":1}
I20260619 02:49:01.286403 5960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.282102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.19.113.67:33979)
0619 02:49:01.282264 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.282268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.282276 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:49:01.282308 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.282310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.282322 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.282387 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.282553 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.283678 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.285171 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.285836 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.286202 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.286264 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.286292 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.286332 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260619 02:49:01.286650 5961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.282269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59374 (local address 127.19.113.65:46253)
0619 02:49:01.282426 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.282431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.282443 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:49:01.282481 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.282483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.282498 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.282596 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.282710 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.283649 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.285190 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.285665 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.285726 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.285777 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.285873 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.286016 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260619 02:49:01.286892 5748 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0fa395f944cdd8271ba6e9c3a2119 (DEFAULT_TABLE table=test-table [id=996d216ebb794d19aa5308d05aa5c9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:01.287009 5882 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0fa395f944cdd8271ba6e9c3a2119 (DEFAULT_TABLE table=test-table [id=996d216ebb794d19aa5308d05aa5c9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:01.287246 5748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0fa395f944cdd8271ba6e9c3a2119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:01.287325 5882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0fa395f944cdd8271ba6e9c3a2119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:01.287281 5614 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0fa395f944cdd8271ba6e9c3a2119 (DEFAULT_TABLE table=test-table [id=996d216ebb794d19aa5308d05aa5c9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:01.287569 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0fa395f944cdd8271ba6e9c3a2119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:01.289840 5962 tablet_bootstrap.cc:492] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Bootstrap starting.
I20260619 02:49:01.290405 5963 tablet_bootstrap.cc:492] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: Bootstrap starting.
I20260619 02:49:01.290684 5962 tablet_bootstrap.cc:654] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:01.290992 5964 tablet_bootstrap.cc:492] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960: Bootstrap starting.
I20260619 02:49:01.291049 5963 tablet_bootstrap.cc:654] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:01.291361 5962 log.cc:824] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:01.291484 5964 tablet_bootstrap.cc:654] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:01.291575 5963 log.cc:824] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:01.291996 5964 log.cc:824] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:01.292125 5962 tablet_bootstrap.cc:492] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: No bootstrap required, opened a new log
I20260619 02:49:01.292191 5962 ts_tablet_manager.cc:1397] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:49:01.292302 5963 tablet_bootstrap.cc:492] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: No bootstrap required, opened a new log
I20260619 02:49:01.292363 5963 ts_tablet_manager.cc:1397] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:49:01.292806 5964 tablet_bootstrap.cc:492] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960: No bootstrap required, opened a new log
I20260619 02:49:01.292870 5964 ts_tablet_manager.cc:1397] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:49:01.293689 5962 raft_consensus.cc:348] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.293836 5962 raft_consensus.cc:374] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:01.293869 5962 raft_consensus.cc:729] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f27d3989baf44adab4cf63d16316b5f, State: Initialized, Role: FOLLOWER
I20260619 02:49:01.293995 5962 consensus_queue.cc:260] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.294246 5962 ts_tablet_manager.cc:1428] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:49:01.294250 5814 heartbeater.cc:503] Master 127.19.113.126:40627 was elected leader, sending a full tablet report...
I20260619 02:49:01.294548 5963 raft_consensus.cc:348] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.294723 5963 raft_consensus.cc:374] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:01.294768 5963 raft_consensus.cc:729] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46e56f9a73342729b80b0ca2185a1c6, State: Initialized, Role: FOLLOWER
I20260619 02:49:01.294672 5964 raft_consensus.cc:348] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.294816 5964 raft_consensus.cc:374] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:01.294903 5964 raft_consensus.cc:729] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730c0382c20d4ffc8e2f8c9f91c32960, State: Initialized, Role: FOLLOWER
I20260619 02:49:01.294914 5963 consensus_queue.cc:260] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.295011 5964 consensus_queue.cc:260] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.295179 5963 ts_tablet_manager.cc:1428] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:49:01.295203 5948 heartbeater.cc:503] Master 127.19.113.126:40627 was elected leader, sending a full tablet report...
I20260619 02:49:01.295238 5964 ts_tablet_manager.cc:1428] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:49:01.295354 5680 heartbeater.cc:503] Master 127.19.113.126:40627 was elected leader, sending a full tablet report...
I20260619 02:49:01.338636 5968 raft_consensus.cc:482] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:49:01.338758 5968 raft_consensus.cc:504] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.339154 5968 leader_election.cc:288] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b46e56f9a73342729b80b0ca2185a1c6 (127.19.113.67:33979), 730c0382c20d4ffc8e2f8c9f91c32960 (127.19.113.65:46253)
I20260619 02:49:01.341789 5960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.339349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:36549 (local address 127.19.113.67:33979)
0619 02:49:01.339404 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.339406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.339415 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:49:01.339474 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.339475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.339486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.339537 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.339689 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.340218 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.341118 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.341485 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.341538 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.341546 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.341705 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.341737 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:49:01.341814 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.339291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:33979 (local address 127.19.113.66:36549)
0619 02:49:01.339352 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.339362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.339407 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.339552 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.339552 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.339555 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.339625 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.339627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.340288 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.340289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.341025 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.341027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.341557 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.341558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.341637 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.341642 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.341698 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.341738 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260619 02:49:01.342005 5961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.339495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:34613 (local address 127.19.113.65:46253)
0619 02:49:01.339545 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.339547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.339555 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:49:01.339577 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.339578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.339588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.339657 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.339796 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.340246 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.341119 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.341650 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.341710 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.341717 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.341903 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.341941 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:49:01.342188 5971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.339295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46253 (local address 127.19.113.66:34613)
0619 02:49:01.339467 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.339477 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.339525 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.339661 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.339662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.339664 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.339733 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.339734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.340318 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.340319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.341039 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.341040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.341758 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.341759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.341824 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.341827 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.342081 (+ 254us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.342118 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":13,"thread_start_us":90,"threads_started":1}
I20260619 02:49:01.342151 5902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0fa395f944cdd8271ba6e9c3a2119" candidate_uuid: "7f27d3989baf44adab4cf63d16316b5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46e56f9a73342729b80b0ca2185a1c6" is_pre_election: true
I20260619 02:49:01.342298 5902 raft_consensus.cc:2454] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f27d3989baf44adab4cf63d16316b5f in term 0.
I20260619 02:49:01.342523 5704 leader_election.cc:302] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f27d3989baf44adab4cf63d16316b5f, b46e56f9a73342729b80b0ca2185a1c6; no voters:
I20260619 02:49:01.342521 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0fa395f944cdd8271ba6e9c3a2119" candidate_uuid: "7f27d3989baf44adab4cf63d16316b5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" is_pre_election: true
I20260619 02:49:01.342660 5968 raft_consensus.cc:2781] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:49:01.342685 5634 raft_consensus.cc:2454] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f27d3989baf44adab4cf63d16316b5f in term 0.
I20260619 02:49:01.342716 5968 raft_consensus.cc:482] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:49:01.342738 5968 raft_consensus.cc:3037] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:01.343511 5968 raft_consensus.cc:504] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.343623 5968 leader_election.cc:288] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [CANDIDATE]: Term 1 election: Requested vote from peers b46e56f9a73342729b80b0ca2185a1c6 (127.19.113.67:33979), 730c0382c20d4ffc8e2f8c9f91c32960 (127.19.113.65:46253)
I20260619 02:49:01.343761 5902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0fa395f944cdd8271ba6e9c3a2119" candidate_uuid: "7f27d3989baf44adab4cf63d16316b5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46e56f9a73342729b80b0ca2185a1c6"
I20260619 02:49:01.343825 5634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0fa395f944cdd8271ba6e9c3a2119" candidate_uuid: "7f27d3989baf44adab4cf63d16316b5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730c0382c20d4ffc8e2f8c9f91c32960"
I20260619 02:49:01.343830 5902 raft_consensus.cc:3037] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:01.343904 5634 raft_consensus.cc:3037] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:01.344555 5902 raft_consensus.cc:2454] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f27d3989baf44adab4cf63d16316b5f in term 1.
I20260619 02:49:01.344555 5634 raft_consensus.cc:2454] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f27d3989baf44adab4cf63d16316b5f in term 1.
I20260619 02:49:01.344724 5704 leader_election.cc:302] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f27d3989baf44adab4cf63d16316b5f, b46e56f9a73342729b80b0ca2185a1c6; no voters:
I20260619 02:49:01.344808 5968 raft_consensus.cc:2781] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:01.344938 5968 raft_consensus.cc:686] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 1 LEADER]: Becoming Leader. State: Replica: 7f27d3989baf44adab4cf63d16316b5f, State: Running, Role: LEADER
I20260619 02:49:01.345023 5968 consensus_queue.cc:237] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } }
I20260619 02:49:01.345836 5488 catalog_manager.cc:5495] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f27d3989baf44adab4cf63d16316b5f (127.19.113.66). New cstate: current_term: 1 leader_uuid: "7f27d3989baf44adab4cf63d16316b5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f27d3989baf44adab4cf63d16316b5f" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 46459 } health_report { overall_health: HEALTHY } } }
W20260619 02:49:01.355804 5815 tablet.cc:2369] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:49:01.361925 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.359669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.19.113.66:46459)
0619 02:49:01.359736 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.359738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.359747 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:49:01.359783 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.359784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.359800 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.359857 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:01.359977 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.360448 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.361204 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.361391 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.361444 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.361601 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:01.361690 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:01.361772 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.361823 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.361834 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:49:01.363722 5902 raft_consensus.cc:1261] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 1 FOLLOWER]: Refusing update from remote peer 7f27d3989baf44adab4cf63d16316b5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:01.363724 5634 raft_consensus.cc:1261] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 1 FOLLOWER]: Refusing update from remote peer 7f27d3989baf44adab4cf63d16316b5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:01.364061 5972 consensus_queue.cc:1035] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b46e56f9a73342729b80b0ca2185a1c6" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 33979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:01.364202 5968 consensus_queue.cc:1035] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "730c0382c20d4ffc8e2f8c9f91c32960" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:01.366817 5975 mvcc.cc:205] Tried to move back new op lower bound from 7298405750222909440 to 7298405750149816320. Current Snapshot: MvccSnapshot[applied={T|T < 7298405750222909440}]
I20260619 02:49:01.367236 5976 mvcc.cc:205] Tried to move back new op lower bound from 7298405750222909440 to 7298405750149816320. Current Snapshot: MvccSnapshot[applied={T|T < 7298405750222909440}]
I20260619 02:49:01.367558 5978 mvcc.cc:205] Tried to move back new op lower bound from 7298405750222909440 to 7298405750149816320. Current Snapshot: MvccSnapshot[applied={T|T < 7298405750222909440}]
I20260619 02:49:01.369158 5488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:49:01.369251 5488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:49:01.369961 5488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4543dcb0138c433e8baf2289f1bbb264: Sending DeleteTablet for 3 replicas of tablet c1c0fa395f944cdd8271ba6e9c3a2119
I20260619 02:49:01.370252 5748 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0fa395f944cdd8271ba6e9c3a2119 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:01 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:40164
I20260619 02:49:01.370255 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0fa395f944cdd8271ba6e9c3a2119 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:01 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:59374
I20260619 02:49:01.370478 5985 tablet_replica.cc:330] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960: stopping tablet replica
I20260619 02:49:01.370486 5882 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0fa395f944cdd8271ba6e9c3a2119 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:01 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:54204
I20260619 02:49:01.370589 5985 raft_consensus.cc:2229] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:49:01.370692 5985 raft_consensus.cc:2258] T c1c0fa395f944cdd8271ba6e9c3a2119 P 730c0382c20d4ffc8e2f8c9f91c32960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:01.370704 5986 tablet_replica.cc:330] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: stopping tablet replica
I20260619 02:49:01.370950 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 5551
I20260619 02:49:01.372352 5986 raft_consensus.cc:2229] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:49:01.372458 5986 raft_consensus.cc:2258] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:01.372864 5986 ts_tablet_manager.cc:1905] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:49:01.374486 5986 ts_tablet_manager.cc:1918] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:49:01.374549 5986 log.cc:1196] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/c1c0fa395f944cdd8271ba6e9c3a2119
I20260619 02:49:01.374791 5986 ts_tablet_manager.cc:1939] T c1c0fa395f944cdd8271ba6e9c3a2119 P 7f27d3989baf44adab4cf63d16316b5f: Deleting consensus metadata
I20260619 02:49:01.375190 5473 catalog_manager.cc:4841] TS 7f27d3989baf44adab4cf63d16316b5f (127.19.113.66:46459): tablet c1c0fa395f944cdd8271ba6e9c3a2119 (table test-table [id=996d216ebb794d19aa5308d05aa5c9df]) successfully deleted
W20260619 02:49:01.377230 5474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:46253 (error 108)
I20260619 02:49:01.377406 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 5685
I20260619 02:49:01.377979 5987 tablet_replica.cc:330] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: stopping tablet replica
I20260619 02:49:01.378011 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.377598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46253 (local address 127.0.0.1:59376)
0619 02:49:01.377947 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.377977 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332}
I20260619 02:49:01.378106 5987 raft_consensus.cc:2229] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:49:01.378209 5987 raft_consensus.cc:2258] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:49:01.378208 5474 catalog_manager.cc:4584] TS 730c0382c20d4ffc8e2f8c9f91c32960 (127.19.113.65:46253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1c0fa395f944cdd8271ba6e9c3a2119: Network error: Client connection negotiation failed: client connection to 127.19.113.65:46253: connect: Connection refused (error 111)
I20260619 02:49:01.378670 5987 ts_tablet_manager.cc:1905] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:49:01.380313 5987 ts_tablet_manager.cc:1918] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:49:01.380386 5987 log.cc:1196] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/c1c0fa395f944cdd8271ba6e9c3a2119
I20260619 02:49:01.380623 5987 ts_tablet_manager.cc:1939] T c1c0fa395f944cdd8271ba6e9c3a2119 P b46e56f9a73342729b80b0ca2185a1c6: Deleting consensus metadata
I20260619 02:49:01.381099 5476 catalog_manager.cc:4841] TS b46e56f9a73342729b80b0ca2185a1c6 (127.19.113.67:33979): tablet c1c0fa395f944cdd8271ba6e9c3a2119 (table test-table [id=996d216ebb794d19aa5308d05aa5c9df]) successfully deleted
I20260619 02:49:01.382920 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 5819
I20260619 02:49:01.387267 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 5457
2026-06-19T02:49:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1782 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T02:49:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:01Z Disabled control of system clock
I20260619 02:49:01.412412 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36413
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:36413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:49:01.490919 5992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:01.491122 5992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:01.491142 5992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:01.492233 5992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:01.492265 5992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:01.492276 5992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:01.492286 5992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:01.492297 5992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:01.493765 5992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:36413
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:36413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:01.494014 5992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:01.494290 5992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:01.498845 5997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:01.498999 5992 server_base.cc:1034] running on GCE node
W20260619 02:49:01.498838 6000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.498836 5998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:01.499383 5992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:01.499652 5992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:01.500783 5992 hybrid_clock.cc:648] HybridClock initialized: now 1781837341500767 us; error 34 us; skew 500 ppm
I20260619 02:49:01.501945 5992 webserver.cc:460] Webserver started at http://127.19.113.126:46453/ using document root <none> and password file <none>
I20260619 02:49:01.502193 5992 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:01.502241 5992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:01.502364 5992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:01.503409 5992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "445bf042fa784a21a32f8ba5b2e857d0"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "4db33e6c9cccf8097ed0d5dbbf60cc9f"
server_key_iv: "91d9177fbb3de626bede66205eb47d0b"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.503726 5992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "445bf042fa784a21a32f8ba5b2e857d0"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "4db33e6c9cccf8097ed0d5dbbf60cc9f"
server_key_iv: "91d9177fbb3de626bede66205eb47d0b"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.504879 5992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:01.505555 6006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.505697 5992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:01.505775 5992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "445bf042fa784a21a32f8ba5b2e857d0"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "4db33e6c9cccf8097ed0d5dbbf60cc9f"
server_key_iv: "91d9177fbb3de626bede66205eb47d0b"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.505843 5992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:01.523556 5992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:01.523828 5992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:01.523927 5992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:01.527631 5992 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:36413
I20260619 02:49:01.527673 6058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:36413 every 8 connection(s)
I20260619 02:49:01.527971 5992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:01.528896 6059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:01.531172 6059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0: Bootstrap starting.
I20260619 02:49:01.531706 6059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:01.531961 6059 log.cc:824] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:01.532560 6059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0: No bootstrap required, opened a new log
I20260619 02:49:01.534116 6059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445bf042fa784a21a32f8ba5b2e857d0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36413 } }
I20260619 02:49:01.534257 6059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:01.534278 6059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445bf042fa784a21a32f8ba5b2e857d0, State: Initialized, Role: FOLLOWER
I20260619 02:49:01.534415 6059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445bf042fa784a21a32f8ba5b2e857d0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36413 } }
I20260619 02:49:01.534475 6059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:01.534495 6059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:01.534539 6059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:01.535069 6059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445bf042fa784a21a32f8ba5b2e857d0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36413 } }
I20260619 02:49:01.535164 6059 leader_election.cc:302] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 445bf042fa784a21a32f8ba5b2e857d0; no voters:
I20260619 02:49:01.535315 6059 leader_election.cc:288] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:01.535357 6062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:01.535522 6062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [term 1 LEADER]: Becoming Leader. State: Replica: 445bf042fa784a21a32f8ba5b2e857d0, State: Running, Role: LEADER
I20260619 02:49:01.535554 6059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:01.535619 6062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445bf042fa784a21a32f8ba5b2e857d0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36413 } }
I20260619 02:49:01.535989 6064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 445bf042fa784a21a32f8ba5b2e857d0. Latest consensus state: current_term: 1 leader_uuid: "445bf042fa784a21a32f8ba5b2e857d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445bf042fa784a21a32f8ba5b2e857d0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36413 } } }
I20260619 02:49:01.536065 6063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "445bf042fa784a21a32f8ba5b2e857d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445bf042fa784a21a32f8ba5b2e857d0" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 36413 } } }
I20260619 02:49:01.536212 6063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:01.536309 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 5992
I20260619 02:49:01.536408 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:01.536604 19909 external_mini_cluster.cc:1428] Setting key 67991446b6e6d22354fafff1954ae6b5
I20260619 02:49:01.536633 6071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:01.536075 6064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:01.537119 6071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:01.538825 6071 catalog_manager.cc:1347] Generated new cluster ID: ca47137775d04b32bd6517b1bf4a490f
I20260619 02:49:01.538894 6071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:01.542781 6081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.537101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.19.113.126:36413)
0619 02:49:01.537364 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.537374 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.537407 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:49:01.537521 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.537527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.537578 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.537697 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:01.538180 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.539447 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.540565 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.541183 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.541286 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.542003 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:01.542007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:01.542007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:01.542199 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:01.542241 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:01.542254 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:01.542310 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.542384 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.542546 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
I20260619 02:49:01.547282 6071 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:01.547739 6071 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:01.550475 6071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0: Generated new TSK 0
I20260619 02:49:01.550650 6071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:01.554308 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36413
--builtin_ntp_servers=127.19.113.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:49:01.632673 6083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:01.632866 6083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:01.632886 6083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:01.634006 6083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:01.634043 6083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:01.634066 6083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:49:01.635494 6083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:01.635703 6083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:01.636045 6083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:01.640725 6088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.640735 6089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.640761 6091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:01.641144 6083 server_base.cc:1034] running on GCE node
I20260619 02:49:01.641325 6083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:01.641527 6083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:01.642669 6083 hybrid_clock.cc:648] HybridClock initialized: now 1781837341642660 us; error 31 us; skew 500 ppm
I20260619 02:49:01.643867 6083 webserver.cc:460] Webserver started at http://127.19.113.65:41017/ using document root <none> and password file <none>
I20260619 02:49:01.644094 6083 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:01.644140 6083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:01.644265 6083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:01.645255 6083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "85d531e4aaf14bf28526a72ca556737a"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "3c415ad725d6784f047c5628222d5690"
server_key_iv: "c881ec48c3d5493517d126809fafe2f6"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.645573 6083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "85d531e4aaf14bf28526a72ca556737a"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "3c415ad725d6784f047c5628222d5690"
server_key_iv: "c881ec48c3d5493517d126809fafe2f6"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.646767 6083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:49:01.647506 6097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.647662 6083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.647739 6083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "85d531e4aaf14bf28526a72ca556737a"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "3c415ad725d6784f047c5628222d5690"
server_key_iv: "c881ec48c3d5493517d126809fafe2f6"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.647804 6083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:01.661580 6083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:01.661841 6083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:01.661959 6083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:01.662159 6083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:01.662444 6083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:01.662477 6083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.662503 6083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:01.662526 6083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.667672 6083 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:45935
I20260619 02:49:01.667742 6210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:45935 every 8 connection(s)
I20260619 02:49:01.668002 6083 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:49:01.668586 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 6083
I20260619 02:49:01.668689 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:49:01.668879 19909 external_mini_cluster.cc:1428] Setting key 166b70fd0ffc52652e567c0208077cba
I20260619 02:49:01.669899 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36413
--builtin_ntp_servers=127.19.113.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:49:01.673208 6081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.668691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:35869 (local address 127.19.113.126:36413)
0619 02:49:01.668743 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.668745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.668996 (+ 251us) server_negotiation.cc:407] Connection header received
0619 02:49:01.669566 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.669567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.669578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.669629 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:01.670451 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.670978 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.671922 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.672137 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.672192 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.672922 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:01.672923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:01.672923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:01.672928 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:01.672942 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:01.672946 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:01.672990 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.673067 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.673098 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:49:01.673398 6213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.668671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36413 (local address 127.19.113.65:35869)
0619 02:49:01.668869 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.668915 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.669479 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.669701 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.669706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.669716 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:01.670368 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.670377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.671069 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.671070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.671844 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.671847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.672205 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.672206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.672327 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.672795 (+ 468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:49:01.672796 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:01.672823 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:49:01.672846 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:01.673001 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:01.673006 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:01.673009 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.673065 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.673171 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260619 02:49:01.673952 6211 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36413
I20260619 02:49:01.674057 6211 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:01.674252 6211 heartbeater.cc:511] Master 127.19.113.126:36413 requested a full tablet report, sending...
I20260619 02:49:01.674757 6023 ts_manager.cc:194] Registered new tserver with Master: 85d531e4aaf14bf28526a72ca556737a (127.19.113.65:45935)
I20260619 02:49:01.675398 6023 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.65:35869
W20260619 02:49:01.748813 6214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:01.749011 6214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:01.749032 6214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:01.750208 6214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:01.750243 6214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:01.750267 6214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:49:01.751626 6214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:01.751840 6214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:01.752097 6214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:01.756860 6220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.756857 6222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.756853 6219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:01.757069 6214 server_base.cc:1034] running on GCE node
I20260619 02:49:01.757282 6214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:01.757507 6214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:01.758653 6214 hybrid_clock.cc:648] HybridClock initialized: now 1781837341758636 us; error 29 us; skew 500 ppm
I20260619 02:49:01.759871 6214 webserver.cc:460] Webserver started at http://127.19.113.66:44567/ using document root <none> and password file <none>
I20260619 02:49:01.760093 6214 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:01.760178 6214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:01.760311 6214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:01.761318 6214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "5d6814ada99e43a69d637741e71ce6d7"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "013f0306a07468009c64e63e9fca5771"
server_key_iv: "5a6618c22d75f316be25479e99e381c3"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.761639 6214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "5d6814ada99e43a69d637741e71ce6d7"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "013f0306a07468009c64e63e9fca5771"
server_key_iv: "5a6618c22d75f316be25479e99e381c3"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.762848 6214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:49:01.763626 6228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.763777 6214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:49:01.763857 6214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "5d6814ada99e43a69d637741e71ce6d7"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "013f0306a07468009c64e63e9fca5771"
server_key_iv: "5a6618c22d75f316be25479e99e381c3"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.763938 6214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:01.781994 6214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:01.782284 6214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:01.782404 6214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:01.782604 6214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:01.782939 6214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:01.782984 6214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.783011 6214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:01.783032 6214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.788324 6214 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:37089
I20260619 02:49:01.788363 6341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:37089 every 8 connection(s)
I20260619 02:49:01.788668 6214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:49:01.793540 6081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.789317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:50297 (local address 127.19.113.126:36413)
0619 02:49:01.789386 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.789388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.789598 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:49:01.790098 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.790100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.790113 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.790166 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:01.790835 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.791351 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.792289 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.792485 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.792540 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.793289 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:01.793290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:01.793291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:01.793295 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:01.793312 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:01.793316 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:01.793359 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.793433 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.793463 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:49:01.793769 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.789303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36413 (local address 127.19.113.66:50297)
0619 02:49:01.789482 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.789518 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.790000 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.790246 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.790251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.790261 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:01.790748 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.790752 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.791435 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.791436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.792207 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.792209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.792557 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.792557 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.792689 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.793160 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:49:01.793160 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:01.793188 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:49:01.793210 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:01.793364 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:01.793368 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:01.793371 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.793432 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.793532 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260619 02:49:01.794374 6342 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36413
I20260619 02:49:01.794430 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 6214
I20260619 02:49:01.794461 6342 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:01.794521 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:49:01.794639 6342 heartbeater.cc:511] Master 127.19.113.126:36413 requested a full tablet report, sending...
I20260619 02:49:01.794719 19909 external_mini_cluster.cc:1428] Setting key 2b15292c8a5e422ab64ecc14b5e07d5b
I20260619 02:49:01.795007 6023 ts_manager.cc:194] Registered new tserver with Master: 5d6814ada99e43a69d637741e71ce6d7 (127.19.113.66:37089)
I20260619 02:49:01.795490 6023 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.66:50297
I20260619 02:49:01.795773 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36413
--builtin_ntp_servers=127.19.113.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:49:01.873764 6345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:01.873966 6345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:01.873986 6345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:01.875118 6345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:01.875156 6345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:01.875180 6345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:49:01.876487 6345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:01.876700 6345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:01.876961 6345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:49:01.881623 6345 server_base.cc:1034] running on GCE node
W20260619 02:49:01.881608 6351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.881608 6353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:01.881649 6350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:01.882004 6345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:01.882218 6345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:01.883352 6345 hybrid_clock.cc:648] HybridClock initialized: now 1781837341883335 us; error 27 us; skew 500 ppm
I20260619 02:49:01.884581 6345 webserver.cc:460] Webserver started at http://127.19.113.67:44447/ using document root <none> and password file <none>
I20260619 02:49:01.884814 6345 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:01.884863 6345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:01.884985 6345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:01.886031 6345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "58fc0a7a524f41d3a4c07eaac41f30ee"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "756168c6857443dd426f6ff3e149562a"
server_key_iv: "83492639d18143df60d820f7eb0b4df8"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.886354 6345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "58fc0a7a524f41d3a4c07eaac41f30ee"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "756168c6857443dd426f6ff3e149562a"
server_key_iv: "83492639d18143df60d820f7eb0b4df8"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.887617 6345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:01.888381 6359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.888538 6345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:49:01.888634 6345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "58fc0a7a524f41d3a4c07eaac41f30ee"
format_stamp: "Formatted at 2026-06-19 02:49:01 on dist-test-slave-zqpb"
server_key: "756168c6857443dd426f6ff3e149562a"
server_key_iv: "83492639d18143df60d820f7eb0b4df8"
server_key_version: "encryptionkey@0"
I20260619 02:49:01.888691 6345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:01.902201 6345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:01.902495 6345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:01.902616 6345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:01.902808 6345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:01.903144 6345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:01.903177 6345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.903204 6345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:01.903221 6345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:01.908492 6345 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:32949
I20260619 02:49:01.908545 6472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:32949 every 8 connection(s)
I20260619 02:49:01.908859 6345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:49:01.910228 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 6345
I20260619 02:49:01.910322 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:49:01.910492 19909 external_mini_cluster.cc:1428] Setting key 5f4b42ecaf5e69f7684545d9cb637c00
I20260619 02:49:01.913810 6081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.909515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:38405 (local address 127.19.113.126:36413)
0619 02:49:01.909581 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.909583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.909802 (+ 219us) server_negotiation.cc:407] Connection header received
0619 02:49:01.910309 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.910310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.910321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.910383 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:01.911151 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.911680 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.912605 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.912799 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.912844 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.913593 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:01.913594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:01.913594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:01.913599 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:01.913612 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:01.913615 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:01.913666 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.913727 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.913753 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:49:01.914004 6475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.909499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:36413 (local address 127.19.113.67:38405)
0619 02:49:01.909682 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.909716 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.910202 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.910478 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.910481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.910489 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:01.911076 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.911082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.911748 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.911749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.912521 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.912523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.912887 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.912888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.913007 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.913450 (+ 443us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:49:01.913450 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:01.913477 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:49:01.913500 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:01.913679 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:01.913682 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:01.913684 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.913728 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.913805 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260619 02:49:01.914378 6473 heartbeater.cc:347] Connected to a master server at 127.19.113.126:36413
I20260619 02:49:01.914458 6473 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:01.914619 6473 heartbeater.cc:511] Master 127.19.113.126:36413 requested a full tablet report, sending...
I20260619 02:49:01.914953 6023 ts_manager.cc:194] Registered new tserver with Master: 58fc0a7a524f41d3a4c07eaac41f30ee (127.19.113.67:32949)
I20260619 02:49:01.915314 6023 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.67:38405
I20260619 02:49:01.922047 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:49:01.925994 6081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.923292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.19.113.126:36413)
0619 02:49:01.923350 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.923352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.923437 (+ 85us) server_negotiation.cc:407] Connection header received
0619 02:49:01.923498 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.923499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.923510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.923560 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:01.923706 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.924175 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.924893 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.925076 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.925126 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.925730 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:01.925731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:01.925732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:01.925736 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:01.925748 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:01.925752 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:01.925821 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.925905 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.925915 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:49:01.927496 6023 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:49:01.928998 6023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:49:01.935144 6482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.931256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38270 (local address 127.19.113.67:32949)
0619 02:49:01.931417 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.931420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.931432 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:49:01.931466 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.931468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.931482 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.931565 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.931749 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.932805 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.934133 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.934769 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.934839 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.934953 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.935050 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.935077 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260619 02:49:01.935171 6481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.931176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:32949 (local address 127.0.0.1:38270)
0619 02:49:01.931298 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.931334 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.931391 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.931633 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.931635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.931637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.931684 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.931685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.932884 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.932885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.934051 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.934053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.934848 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.934849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.934956 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.934970 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.935039 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.935099 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260619 02:49:01.935261 6483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.931368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:45935 (local address 127.0.0.1:35066)
0619 02:49:01.931497 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.931509 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.931550 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.932193 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.932194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.932196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.932221 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.932222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.933316 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.933317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.934294 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.934296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.935069 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.935070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.935123 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.935126 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.935175 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.935209 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260619 02:49:01.935477 6485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.931699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35066 (local address 127.19.113.65:45935)
0619 02:49:01.932020 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.932024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.932033 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:49:01.932069 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.932071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.932084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.932201 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.932261 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.933240 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.934365 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.934888 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.934949 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.935037 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.935368 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.935404 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":73,"threads_started":1}
I20260619 02:49:01.935993 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.931521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:37089 (local address 127.0.0.1:58534)
0619 02:49:01.931865 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.931875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.931931 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.932168 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.932169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.932171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.932210 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.932212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.933061 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.933062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.933954 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.933956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.935781 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.935782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.935838 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.935843 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.935895 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.935930 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":68,"threads_started":1}
I20260619 02:49:01.936168 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.931712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58534 (local address 127.19.113.66:37089)
0619 02:49:01.931880 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.931884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.931921 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:49:01.932012 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.932014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.932029 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.932103 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.932281 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.932983 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.934034 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.934767 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.934839 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.934950 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.935923 (+ 973us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.935966 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260619 02:49:01.936117 6145 tablet_service.cc:1467] Processing CreateTablet for tablet fc4af7b92771450083f4ca5e0a87e8e1 (DEFAULT_TABLE table=test-table [id=7625905ae297444db728718c777f65f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:01.936115 6407 tablet_service.cc:1467] Processing CreateTablet for tablet fc4af7b92771450083f4ca5e0a87e8e1 (DEFAULT_TABLE table=test-table [id=7625905ae297444db728718c777f65f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:01.936461 6407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc4af7b92771450083f4ca5e0a87e8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:01.936461 6145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc4af7b92771450083f4ca5e0a87e8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:01.936834 6276 tablet_service.cc:1467] Processing CreateTablet for tablet fc4af7b92771450083f4ca5e0a87e8e1 (DEFAULT_TABLE table=test-table [id=7625905ae297444db728718c777f65f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:01.937129 6276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc4af7b92771450083f4ca5e0a87e8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:01.938833 6487 tablet_bootstrap.cc:492] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: Bootstrap starting.
I20260619 02:49:01.939610 6488 tablet_bootstrap.cc:492] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a: Bootstrap starting.
I20260619 02:49:01.939651 6487 tablet_bootstrap.cc:654] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:01.939942 6487 log.cc:824] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:01.940203 6488 tablet_bootstrap.cc:654] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:01.940264 6489 tablet_bootstrap.cc:492] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: Bootstrap starting.
I20260619 02:49:01.940488 6488 log.cc:824] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:01.940828 6489 tablet_bootstrap.cc:654] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:01.940945 6487 tablet_bootstrap.cc:492] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: No bootstrap required, opened a new log
I20260619 02:49:01.941010 6487 ts_tablet_manager.cc:1397] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:49:01.941123 6489 log.cc:824] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:01.941226 6488 tablet_bootstrap.cc:492] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a: No bootstrap required, opened a new log
I20260619 02:49:01.941287 6488 ts_tablet_manager.cc:1397] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:49:01.941854 6489 tablet_bootstrap.cc:492] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: No bootstrap required, opened a new log
I20260619 02:49:01.941921 6489 ts_tablet_manager.cc:1397] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:49:01.943250 6487 raft_consensus.cc:348] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.943379 6487 raft_consensus.cc:374] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:01.943409 6487 raft_consensus.cc:729] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58fc0a7a524f41d3a4c07eaac41f30ee, State: Initialized, Role: FOLLOWER
I20260619 02:49:01.943533 6487 consensus_queue.cc:260] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.943768 6487 ts_tablet_manager.cc:1428] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:49:01.943681 6488 raft_consensus.cc:348] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.943819 6488 raft_consensus.cc:374] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:01.943849 6488 raft_consensus.cc:729] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d531e4aaf14bf28526a72ca556737a, State: Initialized, Role: FOLLOWER
I20260619 02:49:01.943883 6473 heartbeater.cc:503] Master 127.19.113.126:36413 was elected leader, sending a full tablet report...
I20260619 02:49:01.943980 6488 consensus_queue.cc:260] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.944228 6488 ts_tablet_manager.cc:1428] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:49:01.944585 6211 heartbeater.cc:503] Master 127.19.113.126:36413 was elected leader, sending a full tablet report...
I20260619 02:49:01.944559 6489 raft_consensus.cc:348] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.944784 6489 raft_consensus.cc:374] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:01.944828 6489 raft_consensus.cc:729] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d6814ada99e43a69d637741e71ce6d7, State: Initialized, Role: FOLLOWER
I20260619 02:49:01.944926 6489 consensus_queue.cc:260] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.945134 6489 ts_tablet_manager.cc:1428] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:49:01.945236 6342 heartbeater.cc:503] Master 127.19.113.126:36413 was elected leader, sending a full tablet report...
I20260619 02:49:01.977540 6495 raft_consensus.cc:482] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:49:01.977681 6495 raft_consensus.cc:504] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.978016 6495 leader_election.cc:288] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58fc0a7a524f41d3a4c07eaac41f30ee (127.19.113.67:32949), 85d531e4aaf14bf28526a72ca556737a (127.19.113.65:45935)
I20260619 02:49:01.980829 6485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.978389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:59169 (local address 127.19.113.65:45935)
0619 02:49:01.978437 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.978439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.978530 (+ 91us) server_negotiation.cc:407] Connection header received
0619 02:49:01.978578 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.978579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.978590 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.978651 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.978778 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.979274 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.980169 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.980525 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.980584 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.980593 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.980726 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.980762 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:49:01.980865 6482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.978307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:33711 (local address 127.19.113.67:32949)
0619 02:49:01.978368 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.978370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.978381 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:49:01.978421 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.978422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.978432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.978477 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:01.978712 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.979261 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.980185 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.980556 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.980615 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.980624 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.980759 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.980798 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:49:01.980870 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.978187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:32949 (local address 127.19.113.66:33711)
0619 02:49:01.978292 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.978306 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.978361 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.978492 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.978492 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.978494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.978605 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.978608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.979341 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.979342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.980098 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.980100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.980646 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.980647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.980697 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.980699 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.980743 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.980783 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":2}
I20260619 02:49:01.980873 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.978273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:45935 (local address 127.19.113.66:59169)
0619 02:49:01.978461 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:01.978470 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:01.978526 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:01.978643 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:01.978644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:01.978645 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:01.978715 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.978716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.979346 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.979347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.980085 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:01.980087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.980598 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.980599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:01.980666 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.980669 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:01.980713 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:49:01.980753 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260619 02:49:01.981215 6427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc4af7b92771450083f4ca5e0a87e8e1" candidate_uuid: "5d6814ada99e43a69d637741e71ce6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" is_pre_election: true
I20260619 02:49:01.981287 6165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc4af7b92771450083f4ca5e0a87e8e1" candidate_uuid: "5d6814ada99e43a69d637741e71ce6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d531e4aaf14bf28526a72ca556737a" is_pre_election: true
I20260619 02:49:01.981369 6427 raft_consensus.cc:2454] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d6814ada99e43a69d637741e71ce6d7 in term 0.
I20260619 02:49:01.981433 6165 raft_consensus.cc:2454] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d6814ada99e43a69d637741e71ce6d7 in term 0.
I20260619 02:49:01.981604 6230 leader_election.cc:302] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58fc0a7a524f41d3a4c07eaac41f30ee, 5d6814ada99e43a69d637741e71ce6d7; no voters:
I20260619 02:49:01.981726 6495 raft_consensus.cc:2781] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:49:01.981797 6495 raft_consensus.cc:482] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:49:01.981820 6495 raft_consensus.cc:3037] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:01.982504 6495 raft_consensus.cc:504] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.982610 6495 leader_election.cc:288] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [CANDIDATE]: Term 1 election: Requested vote from peers 58fc0a7a524f41d3a4c07eaac41f30ee (127.19.113.67:32949), 85d531e4aaf14bf28526a72ca556737a (127.19.113.65:45935)
I20260619 02:49:01.982806 6427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc4af7b92771450083f4ca5e0a87e8e1" candidate_uuid: "5d6814ada99e43a69d637741e71ce6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee"
I20260619 02:49:01.982824 6165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc4af7b92771450083f4ca5e0a87e8e1" candidate_uuid: "5d6814ada99e43a69d637741e71ce6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d531e4aaf14bf28526a72ca556737a"
I20260619 02:49:01.982905 6427 raft_consensus.cc:3037] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:01.982905 6165 raft_consensus.cc:3037] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:01.983750 6427 raft_consensus.cc:2454] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d6814ada99e43a69d637741e71ce6d7 in term 1.
I20260619 02:49:01.983750 6165 raft_consensus.cc:2454] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d6814ada99e43a69d637741e71ce6d7 in term 1.
I20260619 02:49:01.983939 6230 leader_election.cc:302] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58fc0a7a524f41d3a4c07eaac41f30ee, 5d6814ada99e43a69d637741e71ce6d7; no voters:
I20260619 02:49:01.984062 6495 raft_consensus.cc:2781] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:01.984251 6495 raft_consensus.cc:686] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 1 LEADER]: Becoming Leader. State: Replica: 5d6814ada99e43a69d637741e71ce6d7, State: Running, Role: LEADER
I20260619 02:49:01.984341 6495 consensus_queue.cc:237] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } }
I20260619 02:49:01.985146 6021 catalog_manager.cc:5495] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d6814ada99e43a69d637741e71ce6d7 (127.19.113.66). New cstate: current_term: 1 leader_uuid: "5d6814ada99e43a69d637741e71ce6d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d6814ada99e43a69d637741e71ce6d7" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 37089 } health_report { overall_health: HEALTHY } } }
I20260619 02:49:01.991886 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:01.989742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58538 (local address 127.19.113.66:37089)
0619 02:49:01.989787 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:01.989789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:01.989796 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:49:01.989857 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:01.989858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:01.989878 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:01.989926 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:01.990022 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.990490 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.991295 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:01.991475 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:01.991523 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:01.991616 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:01.991694 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:01.991755 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:01.991795 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 02:49:01.991806 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:49:01.993463 6165 raft_consensus.cc:1261] T fc4af7b92771450083f4ca5e0a87e8e1 P 85d531e4aaf14bf28526a72ca556737a [term 1 FOLLOWER]: Refusing update from remote peer 5d6814ada99e43a69d637741e71ce6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:01.993462 6427 raft_consensus.cc:1261] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 1 FOLLOWER]: Refusing update from remote peer 5d6814ada99e43a69d637741e71ce6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:01.993738 6495 consensus_queue.cc:1035] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58fc0a7a524f41d3a4c07eaac41f30ee" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 32949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:01.993863 6497 consensus_queue.cc:1035] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d531e4aaf14bf28526a72ca556737a" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 45935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:01.996357 6501 mvcc.cc:205] Tried to move back new op lower bound from 7298405752803008512 to 7298405752768507904. Current Snapshot: MvccSnapshot[applied={T|T < 7298405752803008512}]
I20260619 02:49:01.996865 6502 mvcc.cc:205] Tried to move back new op lower bound from 7298405752803008512 to 7298405752768507904. Current Snapshot: MvccSnapshot[applied={T|T < 7298405752803008512}]
I20260619 02:49:01.998098 6021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:49:01.998175 6021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:49:01.998857 6021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 445bf042fa784a21a32f8ba5b2e857d0: Sending DeleteTablet for 3 replicas of tablet fc4af7b92771450083f4ca5e0a87e8e1
I20260619 02:49:01.999188 6407 tablet_service.cc:1514] Processing DeleteTablet for tablet fc4af7b92771450083f4ca5e0a87e8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:01 UTC) from {username='slave'} at 127.0.0.1:38270
I20260619 02:49:01.999295 6145 tablet_service.cc:1514] Processing DeleteTablet for tablet fc4af7b92771450083f4ca5e0a87e8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:01 UTC) from {username='slave'} at 127.0.0.1:35066
I20260619 02:49:01.999471 6510 tablet_replica.cc:330] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: stopping tablet replica
I20260619 02:49:01.999457 6276 tablet_service.cc:1514] Processing DeleteTablet for tablet fc4af7b92771450083f4ca5e0a87e8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:01 UTC) from {username='slave'} at 127.0.0.1:58534
I20260619 02:49:01.999610 6510 raft_consensus.cc:2229] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:49:01.999640 6512 tablet_replica.cc:330] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: stopping tablet replica
I20260619 02:49:01.999743 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 6083
I20260619 02:49:01.999768 6510 raft_consensus.cc:2258] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:02.000344 6512 raft_consensus.cc:2229] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:49:02.000473 6512 raft_consensus.cc:2258] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:02.000830 6510 ts_tablet_manager.cc:1905] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:49:02.000870 6512 ts_tablet_manager.cc:1905] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:49:02.002439 6510 ts_tablet_manager.cc:1918] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:49:02.002452 6512 ts_tablet_manager.cc:1918] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:49:02.002511 6512 log.cc:1196] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/fc4af7b92771450083f4ca5e0a87e8e1
I20260619 02:49:02.002605 6510 log.cc:1196] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/fc4af7b92771450083f4ca5e0a87e8e1
I20260619 02:49:02.002758 6512 ts_tablet_manager.cc:1939] T fc4af7b92771450083f4ca5e0a87e8e1 P 5d6814ada99e43a69d637741e71ce6d7: Deleting consensus metadata
I20260619 02:49:02.002974 6510 ts_tablet_manager.cc:1939] T fc4af7b92771450083f4ca5e0a87e8e1 P 58fc0a7a524f41d3a4c07eaac41f30ee: Deleting consensus metadata
I20260619 02:49:02.003329 6008 catalog_manager.cc:4841] TS 5d6814ada99e43a69d637741e71ce6d7 (127.19.113.66:37089): tablet fc4af7b92771450083f4ca5e0a87e8e1 (table test-table [id=7625905ae297444db728718c777f65f2]) successfully deleted
I20260619 02:49:02.003417 6008 catalog_manager.cc:4841] TS 58fc0a7a524f41d3a4c07eaac41f30ee (127.19.113.67:32949): tablet fc4af7b92771450083f4ca5e0a87e8e1 (table test-table [id=7625905ae297444db728718c777f65f2]) successfully deleted
W20260619 02:49:02.005985 6008 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:45935 (error 108)
I20260619 02:49:02.006445 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:02.006318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:45935 (local address 127.0.0.1:35068)
0619 02:49:02.006370 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:02.006410 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:49:02.006466 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 6214
W20260619 02:49:02.007246 6008 catalog_manager.cc:4584] TS 85d531e4aaf14bf28526a72ca556737a (127.19.113.65:45935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc4af7b92771450083f4ca5e0a87e8e1: Network error: Client connection negotiation failed: client connection to 127.19.113.65:45935: connect: Connection refused (error 111)
I20260619 02:49:02.010941 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 6345
I20260619 02:49:02.014978 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 5992
2026-06-19T02:49:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (627 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:02 dist-test-slave-zqpb krb5kdc[6516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:02 dist-test-slave-zqpb krb5kdc[6516](info): set up 2 sockets
Jun 19 02:49:02 dist-test-slave-zqpb krb5kdc[6516](info): commencing operation
krb5kdc: starting...
W20260619 02:49:02.970575 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:03.081312 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:42529
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:42529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:03.159780 6529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:03.159969 6529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:03.159991 6529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:03.161123 6529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:03.161162 6529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:03.161175 6529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:03.161186 6529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:03.161197 6529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:03.162617 6529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:42529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:42529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:03.162863 6529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:03.163086 6529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:03.167464 6534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:03.167596 6529 server_base.cc:1034] running on GCE node
W20260619 02:49:03.167472 6535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:03.167464 6537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:03.167912 6529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:03.168208 6529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:03.169343 6529 hybrid_clock.cc:648] HybridClock initialized: now 1781837343169327 us; error 29 us; skew 500 ppm
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:03.171705 6529 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:03.172109 6529 webserver.cc:460] Webserver started at http://127.19.113.126:42051/ using document root <none> and password file <none>
I20260619 02:49:03.172317 6529 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:03.172364 6529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:03.172485 6529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:03.173457 6529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "65e47e0a698f4395b13d8896924d50bb"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "31ef7ec78ab544b025b33ba143b9d188"
server_key_iv: "8615b7a47e13f3cc198b6ec965afd1b1"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.173776 6529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "65e47e0a698f4395b13d8896924d50bb"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "31ef7ec78ab544b025b33ba143b9d188"
server_key_iv: "8615b7a47e13f3cc198b6ec965afd1b1"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.174969 6529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:03.175617 6544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.175756 6529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:03.175834 6529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "65e47e0a698f4395b13d8896924d50bb"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "31ef7ec78ab544b025b33ba143b9d188"
server_key_iv: "8615b7a47e13f3cc198b6ec965afd1b1"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.175900 6529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:03.191424 6529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:03.193185 6529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:03.193316 6529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:03.197253 6529 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:42529
I20260619 02:49:03.197350 6596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:42529 every 8 connection(s)
I20260619 02:49:03.197649 6529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:03.198515 6597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:03.200799 6597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb: Bootstrap starting.
I20260619 02:49:03.201352 6597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:03.201598 6597 log.cc:824] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:03.202277 6597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb: No bootstrap required, opened a new log
I20260619 02:49:03.203879 6597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e47e0a698f4395b13d8896924d50bb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42529 } }
I20260619 02:49:03.204016 6597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:03.204047 6597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65e47e0a698f4395b13d8896924d50bb, State: Initialized, Role: FOLLOWER
I20260619 02:49:03.204187 6597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e47e0a698f4395b13d8896924d50bb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42529 } }
I20260619 02:49:03.204246 6597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:03.204267 6597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:03.204312 6597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:03.204820 6597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e47e0a698f4395b13d8896924d50bb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42529 } }
I20260619 02:49:03.204906 6597 leader_election.cc:302] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65e47e0a698f4395b13d8896924d50bb; no voters:
I20260619 02:49:03.205065 6597 leader_election.cc:288] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:03.205101 6600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:03.205274 6600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [term 1 LEADER]: Becoming Leader. State: Replica: 65e47e0a698f4395b13d8896924d50bb, State: Running, Role: LEADER
I20260619 02:49:03.205304 6597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:03.205353 6600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e47e0a698f4395b13d8896924d50bb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42529 } }
I20260619 02:49:03.205821 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 6529
I20260619 02:49:03.205925 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:03.206118 19909 external_mini_cluster.cc:1428] Setting key 1bc554eda09f6e9a0f99118b6993fba2
I20260619 02:49:03.206514 6601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65e47e0a698f4395b13d8896924d50bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e47e0a698f4395b13d8896924d50bb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42529 } } }
I20260619 02:49:03.206607 6601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:03.206979 6602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65e47e0a698f4395b13d8896924d50bb. Latest consensus state: current_term: 1 leader_uuid: "65e47e0a698f4395b13d8896924d50bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e47e0a698f4395b13d8896924d50bb" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42529 } } }
I20260619 02:49:03.207561 6602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb [sys.catalog]: This master's current role is: LEADER
W20260619 02:49:03.207160 6616 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:03.207656 6616 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:03.207724 6617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:03.207906 6617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:03.209408 6617 catalog_manager.cc:1347] Generated new cluster ID: ec9ca373596f4458a10a38ed754692a4
I20260619 02:49:03.209458 6617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:03.217298 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.206783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37998 (local address 127.19.113.126:42529)
0619 02:49:03.206941 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.206951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.207057 (+ 106us) server_negotiation.cc:407] Connection header received
0619 02:49:03.207315 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.207320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.207364 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.207478 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:03.208273 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.209442 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.210494 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.211184 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.211298 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.213942 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:03.213945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:03.213946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:03.213958 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:03.215770 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.216140 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.216142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.216142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.216252 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.216434 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.216435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.216435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.216710 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:03.216838 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.216947 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.217072 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260619 02:49:03.227959 6617 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:03.228471 6617 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:03.231189 6617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb: Generated new TSK 0
I20260619 02:49:03.231362 6617 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.113.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
Entry for principal kudu/127.19.113.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab.
I20260619 02:49:03.259328 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42529
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:03.363451 6623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:03.363647 6623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:03.363680 6623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:03.364801 6623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:03.364840 6623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:03.364876 6623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:49:03.366281 6623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.65.keytab
--principal=kudu/127.19.113.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:03.366503 6623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:03.366711 6623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:03.371326 6628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:03.371379 6629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:03.371381 6623 server_base.cc:1034] running on GCE node
W20260619 02:49:03.371335 6631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:03.371675 6623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:03.371882 6623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:03.373008 6623 hybrid_clock.cc:648] HybridClock initialized: now 1781837343372994 us; error 26 us; skew 500 ppm
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:03.375356 6623 init.cc:377] Logged in from keytab as kudu/127.19.113.65@KRBTEST.COM (short username kudu)
I20260619 02:49:03.375751 6623 webserver.cc:460] Webserver started at http://127.19.113.65:45219/ using document root <none> and password file <none>
I20260619 02:49:03.375952 6623 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:03.375998 6623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:03.376118 6623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:03.377087 6623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "db05bb49c9664e37aa889e9a02f6a382"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "bdf4784cab2ac906a5edcc4b11f3357c"
server_key_iv: "7a8dc451e5b0129dbe7b8da41e0317d0"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.377398 6623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "db05bb49c9664e37aa889e9a02f6a382"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "bdf4784cab2ac906a5edcc4b11f3357c"
server_key_iv: "7a8dc451e5b0129dbe7b8da41e0317d0"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.378526 6623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:03.379181 6638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.379321 6623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.379397 6623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "db05bb49c9664e37aa889e9a02f6a382"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "bdf4784cab2ac906a5edcc4b11f3357c"
server_key_iv: "7a8dc451e5b0129dbe7b8da41e0317d0"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.379454 6623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:03.393828 6623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:03.395859 6623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:03.395990 6623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:03.396200 6623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:03.396483 6623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:03.396524 6623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.396554 6623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:03.396577 6623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.401834 6623 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:46525
I20260619 02:49:03.401893 6751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:46525 every 8 connection(s)
I20260619 02:49:03.402159 6623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:49:03.404320 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 6623
I20260619 02:49:03.404429 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:49:03.404630 19909 external_mini_cluster.cc:1428] Setting key 97de52668100e32c8fc7e6613bd91f56
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.65@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:03.411556 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.402816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:46493 (local address 127.19.113.126:42529)
0619 02:49:03.402887 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.402889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.403075 (+ 186us) server_negotiation.cc:407] Connection header received
0619 02:49:03.403538 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.403539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.403549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.403599 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:03.404643 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.405288 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.406450 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.406750 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.406817 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.408860 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:03.408862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:03.408862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:03.408869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:03.410162 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.410666 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.410666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.410666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.410713 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.410988 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.410988 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.410988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.411085 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:03.411131 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.411431 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.411490 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:49:03.411818 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.402795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42529 (local address 127.19.113.65:46493)
0619 02:49:03.402969 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.403002 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:03.403448 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:03.403661 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:03.403665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:03.404255 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:03.404536 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.404542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.405364 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.405365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.406327 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.406330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.406828 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.406829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.406990 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.407433 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:03.407442 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:03.408774 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:03.410282 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:03.410289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:03.410292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:03.410594 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:03.410768 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:03.410769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:03.410770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:03.410923 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:03.411138 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:03.411143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:03.411302 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 02:49:03.411406 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:49:03.411596 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260619 02:49:03.412348 6752 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42529
I20260619 02:49:03.412434 6752 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:03.412596 6752 heartbeater.cc:511] Master 127.19.113.126:42529 requested a full tablet report, sending...
I20260619 02:49:03.413075 6561 ts_manager.cc:194] Registered new tserver with Master: db05bb49c9664e37aa889e9a02f6a382 (127.19.113.65:46525)
WARNING: no policy specified for kudu/127.19.113.66@KRBTEST.COM; defaulting to no policy
I20260619 02:49:03.413666 6561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.65@KRBTEST.COM'} at 127.19.113.65:46493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
Entry for principal kudu/127.19.113.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab.
I20260619 02:49:03.428002 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42529
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:03.506349 6757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:03.506561 6757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:03.506585 6757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:03.507766 6757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:03.507807 6757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:03.507831 6757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:49:03.509243 6757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.66.keytab
--principal=kudu/127.19.113.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:03.509457 6757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:03.509663 6757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:03.514268 6765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:03.514326 6763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:03.514364 6757 server_base.cc:1034] running on GCE node
W20260619 02:49:03.514295 6762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:03.514581 6757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:03.514804 6757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:03.515935 6757 hybrid_clock.cc:648] HybridClock initialized: now 1781837343515931 us; error 32 us; skew 500 ppm
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:03.518221 6757 init.cc:377] Logged in from keytab as kudu/127.19.113.66@KRBTEST.COM (short username kudu)
I20260619 02:49:03.518640 6757 webserver.cc:460] Webserver started at http://127.19.113.66:35003/ using document root <none> and password file <none>
I20260619 02:49:03.518846 6757 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:03.518919 6757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:03.519026 6757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:03.520005 6757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "ec8f5e15c4104969ac36e20aa98a691e"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "e0eef8ab58df799b0068dd9d9786d0ce"
server_key_iv: "e9d56f415a3920ac5793493698b648c8"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.520321 6757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "ec8f5e15c4104969ac36e20aa98a691e"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "e0eef8ab58df799b0068dd9d9786d0ce"
server_key_iv: "e9d56f415a3920ac5793493698b648c8"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.521512 6757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:49:03.522194 6772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.522343 6757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:03.522415 6757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "ec8f5e15c4104969ac36e20aa98a691e"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "e0eef8ab58df799b0068dd9d9786d0ce"
server_key_iv: "e9d56f415a3920ac5793493698b648c8"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.522480 6757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:03.543710 6757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:03.545557 6757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:03.545680 6757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:03.545888 6757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:03.546172 6757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:03.546203 6757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.546224 6757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:03.546247 6757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.551644 6757 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:41861
I20260619 02:49:03.551694 6885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:41861 every 8 connection(s)
I20260619 02:49:03.551980 6757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:49:03.552466 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 6757
I20260619 02:49:03.552559 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:49:03.552759 19909 external_mini_cluster.cc:1428] Setting key cac4d28172f553b12a42f7b7bdacfae4
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.66@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.113.67@KRBTEST.COM; defaulting to no policy
I20260619 02:49:03.561858 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.552712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:43681 (local address 127.19.113.126:42529)
0619 02:49:03.552763 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.552766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.552938 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:49:03.553475 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.553476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.553488 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.553531 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:03.554445 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.554975 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.556183 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.556508 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.556569 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.559228 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:03.559229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:03.559230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:03.559238 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:03.560634 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.560992 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.560993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.560994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.561054 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.561261 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.561262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.561262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.561395 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:03.561466 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.561675 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.561760 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:49:03.561992 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.552649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42529 (local address 127.19.113.66:43681)
0619 02:49:03.552810 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.552852 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:03.553395 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:03.553618 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:03.553622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:03.554171 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:03.554356 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.554361 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.555049 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.555050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.556098 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.556101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.556613 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.556616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.556793 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.557533 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:03.557544 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:03.559141 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:03.560706 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:03.560712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:03.560714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:03.560924 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:03.561123 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:03.561123 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:03.561124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:03.561208 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:03.561454 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:03.561456 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:03.561564 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 02:49:03.561638 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:49:03.561800 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260619 02:49:03.562456 6886 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42529
I20260619 02:49:03.562541 6886 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:03.562705 6886 heartbeater.cc:511] Master 127.19.113.126:42529 requested a full tablet report, sending...
I20260619 02:49:03.563071 6561 ts_manager.cc:194] Registered new tserver with Master: ec8f5e15c4104969ac36e20aa98a691e (127.19.113.66:41861)
I20260619 02:49:03.563448 6561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.66@KRBTEST.COM'} at 127.19.113.66:43681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
Entry for principal kudu/127.19.113.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab.
I20260619 02:49:03.577004 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42529
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:03.654695 6891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:03.654906 6891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:03.654937 6891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:03.656129 6891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:03.656169 6891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:03.656203 6891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:49:03.657575 6891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.67.keytab
--principal=kudu/127.19.113.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:03.657797 6891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:03.657994 6891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:49:03.662575 6891 server_base.cc:1034] running on GCE node
W20260619 02:49:03.662631 6897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:03.662541 6899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:03.662549 6896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:03.662945 6891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:03.663179 6891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:03.664307 6891 hybrid_clock.cc:648] HybridClock initialized: now 1781837343664292 us; error 26 us; skew 500 ppm
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:03.666555 6891 init.cc:377] Logged in from keytab as kudu/127.19.113.67@KRBTEST.COM (short username kudu)
I20260619 02:49:03.666988 6891 webserver.cc:460] Webserver started at http://127.19.113.67:33151/ using document root <none> and password file <none>
I20260619 02:49:03.667182 6891 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:03.667229 6891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:03.667348 6891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:03.668345 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "d8648063a08e4aa2921f8844e07e773f"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "d8daa4d660c0e2ded38a05b63e647bd9"
server_key_iv: "fe9f2a14b5b976d0eee30a33de9b108c"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.668660 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "d8648063a08e4aa2921f8844e07e773f"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "d8daa4d660c0e2ded38a05b63e647bd9"
server_key_iv: "fe9f2a14b5b976d0eee30a33de9b108c"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.669842 6891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:03.670548 6906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.670713 6891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:03.670804 6891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "d8648063a08e4aa2921f8844e07e773f"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "d8daa4d660c0e2ded38a05b63e647bd9"
server_key_iv: "fe9f2a14b5b976d0eee30a33de9b108c"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.670858 6891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:03.703701 6891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:03.705489 6891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:03.705605 6891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:03.705789 6891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:03.706064 6891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:03.706096 6891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.706133 6891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:03.706158 6891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.711637 6891 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:39675
I20260619 02:49:03.711712 7019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:39675 every 8 connection(s)
I20260619 02:49:03.711963 6891 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:49:03 dist-test-slave-zqpb krb5kdc[6516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.67@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:03.719915 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.712602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:56093 (local address 127.19.113.126:42529)
0619 02:49:03.712679 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.712681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.712835 (+ 154us) server_negotiation.cc:407] Connection header received
0619 02:49:03.713322 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.713323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.713332 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.713380 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:03.714215 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.714690 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.715663 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.715844 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.715886 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.717821 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:03.717822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:03.717823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:03.717829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:03.718911 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.719244 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.719245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.719245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.719293 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.719457 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.719457 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.719458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.719547 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:03.719590 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.719798 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.719852 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:49:03.720074 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.712580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42529 (local address 127.19.113.67:56093)
0619 02:49:03.712728 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.712763 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:03.713249 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:03.713432 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:03.713434 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:03.713965 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:03.714153 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.714158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.714758 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.714759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.715592 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.715594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.715896 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.715897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.715989 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.716395 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:03.716403 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:03.717746 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:03.718981 (+ 1235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:03.718987 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:03.718989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:03.719192 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:03.719336 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:03.719336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:03.719337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:03.719419 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:03.719588 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:03.719591 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:03.719704 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 02:49:03.719780 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:49:03.719887 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260619 02:49:03.720456 7020 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42529
I20260619 02:49:03.720530 7020 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:03.720666 7020 heartbeater.cc:511] Master 127.19.113.126:42529 requested a full tablet report, sending...
I20260619 02:49:03.720952 6561 ts_manager.cc:194] Registered new tserver with Master: d8648063a08e4aa2921f8844e07e773f (127.19.113.67:39675)
I20260619 02:49:03.721330 6561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.113.67@KRBTEST.COM'} at 127.19.113.67:56093
I20260619 02:49:03.721423 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 6891
I20260619 02:49:03.721520 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:49:03.721696 19909 external_mini_cluster.cc:1428] Setting key f2f08efc4aeac8f4f9a02f9c144e51f3
I20260619 02:49:03.722342 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:49:03.728895 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.723539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.19.113.126:42529)
0619 02:49:03.723592 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.723594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.723679 (+ 85us) server_negotiation.cc:407] Connection header received
0619 02:49:03.723736 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.723737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.723746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.723786 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:03.724274 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.724715 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.725452 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.725641 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.725691 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.726760 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:03.726761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:03.726762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:03.726769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:03.727887 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.728197 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.728198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.728198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.728251 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:03.728402 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:03.728402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:03.728403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:03.728536 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:03.728619 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.728740 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.728810 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:49:03.730360 6561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:49:03.732129 6561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:49:03.738082 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.734555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.19.113.67:39675)
0619 02:49:03.734732 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.734736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.734746 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:49:03.734780 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.734782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.734795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.734859 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:03.735045 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.735814 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.737084 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.737721 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.737780 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.737835 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.737969 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.738006 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260619 02:49:03.738072 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.734213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:39675 (local address 127.0.0.1:44064)
0619 02:49:03.734675 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.734683 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:03.734715 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:03.734896 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:03.734897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:03.734899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:03.734975 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.734977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.735898 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.735899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.736994 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.736997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.737790 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.737791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.737873 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.737886 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:49:03.737947 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:49:03.737996 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":88,"threads_started":1}
I20260619 02:49:03.738274 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.734210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46525 (local address 127.0.0.1:41470)
0619 02:49:03.734558 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.734566 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:03.734614 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:03.734889 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:03.734892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:03.734894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:03.735084 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.735086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.736353 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.736354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.737208 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.737210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.738054 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.738055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.738117 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.738121 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:49:03.738168 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:49:03.738205 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":56,"thread_start_us":94,"threads_started":1}
I20260619 02:49:03.738554 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.734389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.19.113.65:46525)
0619 02:49:03.734539 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.734544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.734608 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:49:03.734671 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.734673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.734689 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.734768 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:03.735170 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.736243 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.737283 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.737900 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.737971 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.738021 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.738437 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.738477 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260619 02:49:03.738628 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.734206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:41861 (local address 127.0.0.1:43760)
0619 02:49:03.734366 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.734405 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:03.734472 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:03.734950 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:03.734951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:03.734953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:03.734990 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.734992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.736343 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.736344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.737430 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.737432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.738451 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.738452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.738507 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.738511 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:49:03.738538 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 02:49:03.738571 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":25,"thread_start_us":40,"threads_started":1}
I20260619 02:49:03.738994 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.734389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.19.113.66:41861)
0619 02:49:03.734759 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.734763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.734772 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:49:03.734806 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.734808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.734822 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.735060 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:03.735143 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.736243 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.737505 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.738366 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.738769 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.738832 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.738857 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.738932 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":89,"threads_started":1}
I20260619 02:49:03.738950 6954 tablet_service.cc:1467] Processing CreateTablet for tablet 50bc58b0d59c4d1982bcd21b52b9cb91 (DEFAULT_TABLE table=test-table [id=5dcdbb4a93b64090afc4ed92ef515fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:03.739305 6954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bc58b0d59c4d1982bcd21b52b9cb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:03.739271 6685 tablet_service.cc:1467] Processing CreateTablet for tablet 50bc58b0d59c4d1982bcd21b52b9cb91 (DEFAULT_TABLE table=test-table [id=5dcdbb4a93b64090afc4ed92ef515fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:03.739555 6685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bc58b0d59c4d1982bcd21b52b9cb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:03.742489 7038 tablet_bootstrap.cc:492] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382: Bootstrap starting.
I20260619 02:49:03.742841 6820 tablet_service.cc:1467] Processing CreateTablet for tablet 50bc58b0d59c4d1982bcd21b52b9cb91 (DEFAULT_TABLE table=test-table [id=5dcdbb4a93b64090afc4ed92ef515fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:03.743008 7037 tablet_bootstrap.cc:492] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: Bootstrap starting.
I20260619 02:49:03.743175 6820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bc58b0d59c4d1982bcd21b52b9cb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:03.743176 7038 tablet_bootstrap.cc:654] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:03.743638 7037 tablet_bootstrap.cc:654] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:03.743755 7038 log.cc:824] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:03.744128 7037 log.cc:824] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:03.744488 7038 tablet_bootstrap.cc:492] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382: No bootstrap required, opened a new log
I20260619 02:49:03.744561 7038 ts_tablet_manager.cc:1397] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:49:03.744752 7037 tablet_bootstrap.cc:492] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: No bootstrap required, opened a new log
I20260619 02:49:03.744808 7037 ts_tablet_manager.cc:1397] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:49:03.746016 7041 tablet_bootstrap.cc:492] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: Bootstrap starting.
I20260619 02:49:03.746384 7038 raft_consensus.cc:348] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.746516 7038 raft_consensus.cc:374] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:03.746400 7037 raft_consensus.cc:348] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.746537 7038 raft_consensus.cc:729] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db05bb49c9664e37aa889e9a02f6a382, State: Initialized, Role: FOLLOWER
I20260619 02:49:03.746551 7037 raft_consensus.cc:374] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:03.746595 7037 raft_consensus.cc:729] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8648063a08e4aa2921f8844e07e773f, State: Initialized, Role: FOLLOWER
I20260619 02:49:03.746677 7038 consensus_queue.cc:260] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.746706 7037 consensus_queue.cc:260] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.746748 7041 tablet_bootstrap.cc:654] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:03.746970 7038 ts_tablet_manager.cc:1428] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:49:03.746989 7037 ts_tablet_manager.cc:1428] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:49:03.746999 7020 heartbeater.cc:503] Master 127.19.113.126:42529 was elected leader, sending a full tablet report...
I20260619 02:49:03.747262 6752 heartbeater.cc:503] Master 127.19.113.126:42529 was elected leader, sending a full tablet report...
I20260619 02:49:03.747409 7041 log.cc:824] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:03.748051 7041 tablet_bootstrap.cc:492] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: No bootstrap required, opened a new log
I20260619 02:49:03.748114 7041 ts_tablet_manager.cc:1397] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:49:03.749361 7041 raft_consensus.cc:348] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.749478 7041 raft_consensus.cc:374] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:03.749506 7041 raft_consensus.cc:729] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8f5e15c4104969ac36e20aa98a691e, State: Initialized, Role: FOLLOWER
I20260619 02:49:03.749712 7041 consensus_queue.cc:260] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.749972 7041 ts_tablet_manager.cc:1428] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:49:03.750015 6886 heartbeater.cc:503] Master 127.19.113.126:42529 was elected leader, sending a full tablet report...
I20260619 02:49:03.753919 7043 raft_consensus.cc:482] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:49:03.753985 7043 raft_consensus.cc:504] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.754241 7043 leader_election.cc:288] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec8f5e15c4104969ac36e20aa98a691e (127.19.113.66:41861), db05bb49c9664e37aa889e9a02f6a382 (127.19.113.65:46525)
I20260619 02:49:03.757517 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.754363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46525 (local address 127.19.113.67:34975)
0619 02:49:03.754403 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.754414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:03.754448 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:03.754559 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:03.754560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:03.754561 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:03.754628 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.754630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.755542 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.755543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.756667 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.756669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.757326 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.757327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.757379 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.757383 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:49:03.757421 (+ 38us) client_negotiation.cc:240] Negotiation successful
0619 02:49:03.757451 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 02:49:03.757534 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.754401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:34975 (local address 127.19.113.65:46525)
0619 02:49:03.754438 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.754439 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.754447 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:49:03.754494 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.754495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.754504 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.754556 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:03.754715 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.755469 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.756737 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.757255 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.757326 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.757334 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.757436 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.757470 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:49:03.757722 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.754432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:34897 (local address 127.19.113.66:41861)
0619 02:49:03.754468 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.754469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.754699 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:49:03.754743 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.754744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.754753 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.754801 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:03.755025 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.755662 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.756861 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.757420 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.757479 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.757486 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.757630 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.757669 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:49:03.757797 6706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bc58b0d59c4d1982bcd21b52b9cb91" candidate_uuid: "d8648063a08e4aa2921f8844e07e773f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db05bb49c9664e37aa889e9a02f6a382" is_pre_election: true
I20260619 02:49:03.757915 6706 raft_consensus.cc:2454] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8648063a08e4aa2921f8844e07e773f in term 0.
I20260619 02:49:03.757941 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.754364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:41861 (local address 127.19.113.67:34897)
0619 02:49:03.754639 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.754648 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:03.754696 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:03.754850 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:03.754851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:03.754852 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:03.754940 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.754942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.755725 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.755725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.756801 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:03.756803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.757485 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.757486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:03.757545 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.757548 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:03.757828 (+ 280us) client_negotiation.cc:240] Negotiation successful
0619 02:49:03.757868 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":134,"thread_start_us":87,"threads_started":1}
I20260619 02:49:03.758136 6908 leader_election.cc:302] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8648063a08e4aa2921f8844e07e773f, db05bb49c9664e37aa889e9a02f6a382; no voters:
I20260619 02:49:03.758224 6840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bc58b0d59c4d1982bcd21b52b9cb91" candidate_uuid: "d8648063a08e4aa2921f8844e07e773f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8f5e15c4104969ac36e20aa98a691e" is_pre_election: true
I20260619 02:49:03.758340 6840 raft_consensus.cc:2454] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8648063a08e4aa2921f8844e07e773f in term 0.
I20260619 02:49:03.758441 7043 raft_consensus.cc:2781] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:49:03.758496 7043 raft_consensus.cc:482] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:49:03.758518 7043 raft_consensus.cc:3037] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:03.759132 7043 raft_consensus.cc:504] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.759234 7043 leader_election.cc:288] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [CANDIDATE]: Term 1 election: Requested vote from peers ec8f5e15c4104969ac36e20aa98a691e (127.19.113.66:41861), db05bb49c9664e37aa889e9a02f6a382 (127.19.113.65:46525)
I20260619 02:49:03.759394 6706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bc58b0d59c4d1982bcd21b52b9cb91" candidate_uuid: "d8648063a08e4aa2921f8844e07e773f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db05bb49c9664e37aa889e9a02f6a382"
I20260619 02:49:03.759440 6840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bc58b0d59c4d1982bcd21b52b9cb91" candidate_uuid: "d8648063a08e4aa2921f8844e07e773f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8f5e15c4104969ac36e20aa98a691e"
I20260619 02:49:03.759455 6706 raft_consensus.cc:3037] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:03.759508 6840 raft_consensus.cc:3037] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:03.760073 6706 raft_consensus.cc:2454] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8648063a08e4aa2921f8844e07e773f in term 1.
I20260619 02:49:03.760090 6840 raft_consensus.cc:2454] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8648063a08e4aa2921f8844e07e773f in term 1.
I20260619 02:49:03.760218 6908 leader_election.cc:302] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8648063a08e4aa2921f8844e07e773f, db05bb49c9664e37aa889e9a02f6a382; no voters:
I20260619 02:49:03.760300 7043 raft_consensus.cc:2781] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:03.760412 7043 raft_consensus.cc:686] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 1 LEADER]: Becoming Leader. State: Replica: d8648063a08e4aa2921f8844e07e773f, State: Running, Role: LEADER
I20260619 02:49:03.760480 7043 consensus_queue.cc:237] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } }
I20260619 02:49:03.761193 6561 catalog_manager.cc:5495] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f reported cstate change: term changed from 0 to 1, leader changed from <none> to d8648063a08e4aa2921f8844e07e773f (127.19.113.67). New cstate: current_term: 1 leader_uuid: "d8648063a08e4aa2921f8844e07e773f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8648063a08e4aa2921f8844e07e773f" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 39675 } health_report { overall_health: HEALTHY } } }
I20260619 02:49:03.772454 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.770463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.19.113.67:39675)
0619 02:49:03.770510 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.770512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.770519 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:49:03.770570 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.770571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.770586 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.770644 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:03.770744 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.771223 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.771887 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.772056 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.772095 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.772212 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:03.772298 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:03.772348 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.772359 (+ 11us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.772374 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:49:03.774096 6840 raft_consensus.cc:1261] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 1 FOLLOWER]: Refusing update from remote peer d8648063a08e4aa2921f8844e07e773f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:03.774096 6706 raft_consensus.cc:1261] T 50bc58b0d59c4d1982bcd21b52b9cb91 P db05bb49c9664e37aa889e9a02f6a382 [term 1 FOLLOWER]: Refusing update from remote peer d8648063a08e4aa2921f8844e07e773f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:03.774438 7047 consensus_queue.cc:1035] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db05bb49c9664e37aa889e9a02f6a382" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 46525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:03.774557 7043 consensus_queue.cc:1035] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8f5e15c4104969ac36e20aa98a691e" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 41861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:03.776671 7050 mvcc.cc:205] Tried to move back new op lower bound from 7298405760096370688 to 7298405760043528192. Current Snapshot: MvccSnapshot[applied={T|T < 7298405760096370688}]
I20260619 02:49:03.777522 7051 mvcc.cc:205] Tried to move back new op lower bound from 7298405760096370688 to 7298405760043528192. Current Snapshot: MvccSnapshot[applied={T|T < 7298405760096370688 or (T in {7298405760096370688})}]
I20260619 02:49:03.779294 6560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:49:03.779389 6560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:49:03.780076 6560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65e47e0a698f4395b13d8896924d50bb: Sending DeleteTablet for 3 replicas of tablet 50bc58b0d59c4d1982bcd21b52b9cb91
I20260619 02:49:03.780391 6685 tablet_service.cc:1514] Processing DeleteTablet for tablet 50bc58b0d59c4d1982bcd21b52b9cb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:03 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:41470
I20260619 02:49:03.780498 6954 tablet_service.cc:1514] Processing DeleteTablet for tablet 50bc58b0d59c4d1982bcd21b52b9cb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:03 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:44064
I20260619 02:49:03.780591 6820 tablet_service.cc:1514] Processing DeleteTablet for tablet 50bc58b0d59c4d1982bcd21b52b9cb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:03 UTC) from {username='kudu', principal='kudu/127.19.113.126@KRBTEST.COM'} at 127.0.0.1:43760
I20260619 02:49:03.780685 7060 tablet_replica.cc:330] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: stopping tablet replica
I20260619 02:49:03.780774 7060 raft_consensus.cc:2229] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:49:03.780788 7061 tablet_replica.cc:330] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: stopping tablet replica
I20260619 02:49:03.780900 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 6623
I20260619 02:49:03.781471 7060 raft_consensus.cc:2258] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:03.781903 7060 ts_tablet_manager.cc:1905] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:49:03.783504 7060 ts_tablet_manager.cc:1918] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:49:03.783576 7060 log.cc:1196] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/50bc58b0d59c4d1982bcd21b52b9cb91
I20260619 02:49:03.783531 7061 raft_consensus.cc:2229] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:49:03.783797 7061 raft_consensus.cc:2258] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:03.783825 7060 ts_tablet_manager.cc:1939] T 50bc58b0d59c4d1982bcd21b52b9cb91 P d8648063a08e4aa2921f8844e07e773f: Deleting consensus metadata
I20260619 02:49:03.784264 7061 ts_tablet_manager.cc:1905] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:49:03.785665 7061 ts_tablet_manager.cc:1918] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:49:03.785735 7061 log.cc:1196] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/50bc58b0d59c4d1982bcd21b52b9cb91
I20260619 02:49:03.785966 7061 ts_tablet_manager.cc:1939] T 50bc58b0d59c4d1982bcd21b52b9cb91 P ec8f5e15c4104969ac36e20aa98a691e: Deleting consensus metadata
I20260619 02:49:03.786320 6545 catalog_manager.cc:4841] TS ec8f5e15c4104969ac36e20aa98a691e (127.19.113.66:41861): tablet 50bc58b0d59c4d1982bcd21b52b9cb91 (table test-table [id=5dcdbb4a93b64090afc4ed92ef515fe4]) successfully deleted
I20260619 02:49:03.786525 6548 catalog_manager.cc:4841] TS d8648063a08e4aa2921f8844e07e773f (127.19.113.67:39675): tablet 50bc58b0d59c4d1982bcd21b52b9cb91 (table test-table [id=5dcdbb4a93b64090afc4ed92ef515fe4]) successfully deleted
W20260619 02:49:03.786684 6546 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:46525 (error 108)
I20260619 02:49:03.786926 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 6757
I20260619 02:49:03.787446 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.787036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:46525 (local address 127.0.0.1:41478)
0619 02:49:03.787375 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:03.787413 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:46525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320}
W20260619 02:49:03.787585 6546 catalog_manager.cc:4584] TS db05bb49c9664e37aa889e9a02f6a382 (127.19.113.65:46525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50bc58b0d59c4d1982bcd21b52b9cb91: Network error: Client connection negotiation failed: client connection to 127.19.113.65:46525: connect: Connection refused (error 111)
I20260619 02:49:03.792546 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 6891
I20260619 02:49:03.797317 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 6529
2026-06-19T02:49:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1782 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T02:49:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:03Z Disabled control of system clock
I20260619 02:49:03.821976 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:42443
--webserver_interface=127.19.113.126
--webserver_port=0
--builtin_ntp_servers=127.19.113.84:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:42443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:49:03.900029 7066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:03.900243 7066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:03.900262 7066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:03.901361 7066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:03.901397 7066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:03.901408 7066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:03.901420 7066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:03.901430 7066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:03.902729 7066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:42443
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:42443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:03.902978 7066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:03.903231 7066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:03.907675 7071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:03.907673 7074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:03.907680 7072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:03.907963 7066 server_base.cc:1034] running on GCE node
I20260619 02:49:03.908191 7066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:03.908448 7066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:03.909582 7066 hybrid_clock.cc:648] HybridClock initialized: now 1781837343909566 us; error 30 us; skew 500 ppm
I20260619 02:49:03.910717 7066 webserver.cc:460] Webserver started at http://127.19.113.126:44209/ using document root <none> and password file <none>
I20260619 02:49:03.910974 7066 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:03.911021 7066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:03.911154 7066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:03.912176 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "6031adcf1b91462789184e0ae562d00c"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "38e6a8773bb81807fac8fd819647c136"
server_key_iv: "9f9ec7b47935178c5b98ed4559859120"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.912500 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "6031adcf1b91462789184e0ae562d00c"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "38e6a8773bb81807fac8fd819647c136"
server_key_iv: "9f9ec7b47935178c5b98ed4559859120"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.913683 7066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:49:03.914351 7080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:03.914475 7066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:49:03.914553 7066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "6031adcf1b91462789184e0ae562d00c"
format_stamp: "Formatted at 2026-06-19 02:49:03 on dist-test-slave-zqpb"
server_key: "38e6a8773bb81807fac8fd819647c136"
server_key_iv: "9f9ec7b47935178c5b98ed4559859120"
server_key_version: "encryptionkey@0"
I20260619 02:49:03.914619 7066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:03.933743 7066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:03.934036 7066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:03.934185 7066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:03.937767 7066 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:42443
I20260619 02:49:03.937808 7132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:42443 every 8 connection(s)
I20260619 02:49:03.938094 7066 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:03.939003 7133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:03.941171 7133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c: Bootstrap starting.
I20260619 02:49:03.941671 7133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:03.941898 7133 log.cc:824] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:03.942471 7133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c: No bootstrap required, opened a new log
I20260619 02:49:03.944060 7133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6031adcf1b91462789184e0ae562d00c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42443 } }
I20260619 02:49:03.944201 7133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:03.944223 7133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6031adcf1b91462789184e0ae562d00c, State: Initialized, Role: FOLLOWER
I20260619 02:49:03.944329 7133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6031adcf1b91462789184e0ae562d00c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42443 } }
I20260619 02:49:03.944398 7133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:03.944420 7133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:03.944451 7133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:03.944914 7133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6031adcf1b91462789184e0ae562d00c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42443 } }
I20260619 02:49:03.944998 7133 leader_election.cc:302] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6031adcf1b91462789184e0ae562d00c; no voters:
I20260619 02:49:03.945118 7133 leader_election.cc:288] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:03.945176 7136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:03.945300 7136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [term 1 LEADER]: Becoming Leader. State: Replica: 6031adcf1b91462789184e0ae562d00c, State: Running, Role: LEADER
I20260619 02:49:03.945392 7133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:03.945392 7136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6031adcf1b91462789184e0ae562d00c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42443 } }
I20260619 02:49:03.946035 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 7066
I20260619 02:49:03.945987 7137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6031adcf1b91462789184e0ae562d00c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6031adcf1b91462789184e0ae562d00c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42443 } } }
I20260619 02:49:03.946135 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:03.946099 7137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:03.946322 19909 external_mini_cluster.cc:1428] Setting key 12cc825d1192322dd0e2d7abbc6deb1c
I20260619 02:49:03.946653 7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6031adcf1b91462789184e0ae562d00c. Latest consensus state: current_term: 1 leader_uuid: "6031adcf1b91462789184e0ae562d00c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6031adcf1b91462789184e0ae562d00c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 42443 } } }
I20260619 02:49:03.946729 7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c [sys.catalog]: This master's current role is: LEADER
W20260619 02:49:03.947412 7151 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:03.947466 7151 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:03.947618 7145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:03.947770 7145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:03.949308 7145 catalog_manager.cc:1347] Generated new cluster ID: aa292a95996247b29343795f8e813b29
I20260619 02:49:03.949365 7145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:03.951752 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:03.946992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39240 (local address 127.19.113.126:42443)
0619 02:49:03.947170 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:03.947180 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:03.947399 (+ 219us) server_negotiation.cc:407] Connection header received
0619 02:49:03.947539 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:03.947546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:03.947621 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:03.947752 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:03.948049 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.949165 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.949987 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:03.950405 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:03.950511 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:03.951169 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:03.951172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:03.951172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:03.951341 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:03.951372 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:03.951383 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:03.951436 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:03.951506 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:49:03.951582 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260619 02:49:03.979362 7145 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:03.979880 7145 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:03.986940 7145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c: Generated new TSK 0
I20260619 02:49:03.987128 7145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:03.991477 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.65:0
--local_ip_for_outbound_sockets=127.19.113.65
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42443
--builtin_ntp_servers=127.19.113.84:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:49:04.071195 7157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:04.071424 7157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:04.071457 7157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:04.072654 7157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:04.072691 7157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:04.072718 7157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.65
I20260619 02:49:04.074126 7157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.113.65
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.65
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:04.074342 7157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:04.074615 7157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:04.081095 7162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:04.081092 7163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:04.081624 7165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:04.082011 7157 server_base.cc:1034] running on GCE node
I20260619 02:49:04.082226 7157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:04.082479 7157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:04.083602 7157 hybrid_clock.cc:648] HybridClock initialized: now 1781837344083578 us; error 37 us; skew 500 ppm
I20260619 02:49:04.084947 7157 webserver.cc:460] Webserver started at http://127.19.113.65:42945/ using document root <none> and password file <none>
I20260619 02:49:04.085162 7157 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:04.085210 7157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:04.085335 7157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:04.086472 7157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/data/instance:
uuid: "2c59da9220894887b2d0e709b280df75"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "6b77a1e04110c7632a8fc55c6c20f49e"
server_key_iv: "bce44c3b6c1a429d3999725d07e464dd"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.086830 7157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance:
uuid: "2c59da9220894887b2d0e709b280df75"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "6b77a1e04110c7632a8fc55c6c20f49e"
server_key_iv: "bce44c3b6c1a429d3999725d07e464dd"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.088511 7157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:49:04.089406 7171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.089610 7157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:49:04.089704 7157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/wal
uuid: "2c59da9220894887b2d0e709b280df75"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "6b77a1e04110c7632a8fc55c6c20f49e"
server_key_iv: "bce44c3b6c1a429d3999725d07e464dd"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.089797 7157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:04.110778 7157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:04.111151 7157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:04.111285 7157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:04.111527 7157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:04.111902 7157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:04.111943 7157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.111974 7157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:04.112000 7157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.118844 7157 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.65:34145
I20260619 02:49:04.118916 7284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.65:34145 every 8 connection(s)
I20260619 02:49:04.119271 7157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/data/info.pb
I20260619 02:49:04.124676 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.119933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.65:59441 (local address 127.19.113.126:42443)
0619 02:49:04.119990 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.119992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.120304 (+ 312us) server_negotiation.cc:407] Connection header received
0619 02:49:04.120957 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.120958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.120972 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.121033 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:04.121884 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.122439 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.123528 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.123722 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.123775 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.124469 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:04.124471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:04.124471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:04.124477 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:04.124491 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:04.124494 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:04.124531 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.124589 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.124615 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:49:04.124855 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.119950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42443 (local address 127.19.113.65:59441)
0619 02:49:04.120153 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.120202 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:04.120856 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:04.121110 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:04.121115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:04.121125 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:04.121799 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.121806 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.122527 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.122528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.123451 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.123454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.123786 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.123786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.123891 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.124353 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:49:04.124353 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:04.124379 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:49:04.124400 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:04.124536 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:04.124538 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:04.124540 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:49:04.124582 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:49:04.124659 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260619 02:49:04.125312 7285 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42443
I20260619 02:49:04.125396 7285 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:04.125564 7285 heartbeater.cc:511] Master 127.19.113.126:42443 requested a full tablet report, sending...
I20260619 02:49:04.126075 7097 ts_manager.cc:194] Registered new tserver with Master: 2c59da9220894887b2d0e709b280df75 (127.19.113.65:34145)
I20260619 02:49:04.126418 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 7157
I20260619 02:49:04.126518 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-0/wal/instance
I20260619 02:49:04.126719 19909 external_mini_cluster.cc:1428] Setting key 415d8bca6b3aed4900a5ef76460adeb4
I20260619 02:49:04.127029 7097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.65:59441
I20260619 02:49:04.127799 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.66:0
--local_ip_for_outbound_sockets=127.19.113.66
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42443
--builtin_ntp_servers=127.19.113.84:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:49:04.205693 7288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:04.205881 7288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:04.205900 7288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:04.207001 7288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:04.207039 7288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:04.207063 7288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.66
I20260619 02:49:04.208338 7288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.113.66
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.66
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:04.208506 7288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:04.208739 7288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:04.213372 7294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:04.213420 7293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:04.213553 7288 server_base.cc:1034] running on GCE node
W20260619 02:49:04.213379 7296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:04.213778 7288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:04.213963 7288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:04.215092 7288 hybrid_clock.cc:648] HybridClock initialized: now 1781837344215084 us; error 30 us; skew 500 ppm
I20260619 02:49:04.216185 7288 webserver.cc:460] Webserver started at http://127.19.113.66:38841/ using document root <none> and password file <none>
I20260619 02:49:04.216398 7288 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:04.216444 7288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:04.216588 7288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:04.217571 7288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/data/instance:
uuid: "f982821780fe4caeb96f887ae8631cb3"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "4cfcd04310c84f85476929fbb26494a8"
server_key_iv: "6a8d972dbc0860ffd204b0312228cbb0"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.217875 7288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance:
uuid: "f982821780fe4caeb96f887ae8631cb3"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "4cfcd04310c84f85476929fbb26494a8"
server_key_iv: "6a8d972dbc0860ffd204b0312228cbb0"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.219079 7288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:49:04.219679 7302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.219825 7288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.219898 7288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/wal
uuid: "f982821780fe4caeb96f887ae8631cb3"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "4cfcd04310c84f85476929fbb26494a8"
server_key_iv: "6a8d972dbc0860ffd204b0312228cbb0"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.219955 7288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:04.225924 7288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:04.226171 7288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:04.226289 7288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:04.226475 7288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:04.226761 7288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:04.226794 7288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.226820 7288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:04.226842 7288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.232226 7288 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.66:44363
I20260619 02:49:04.232265 7415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.66:44363 every 8 connection(s)
I20260619 02:49:04.232569 7288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/data/info.pb
I20260619 02:49:04.237305 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.233293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.66:49827 (local address 127.19.113.126:42443)
0619 02:49:04.233348 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.233350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.233496 (+ 146us) server_negotiation.cc:407] Connection header received
0619 02:49:04.233957 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.233958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.233966 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.234021 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:04.234618 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.235139 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.236059 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.236253 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.236300 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.237092 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:04.237093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:04.237094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:04.237097 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:04.237114 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:04.237119 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:04.237155 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.237222 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.237247 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:49:04.237498 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.233217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42443 (local address 127.19.113.66:49827)
0619 02:49:04.233389 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.233429 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:04.233881 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:04.234077 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:04.234080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:04.234088 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:04.234548 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.234552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.235213 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.235215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.235993 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.235996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.236321 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.236322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.236468 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.236961 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:49:04.236962 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:04.236989 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:49:04.237012 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:04.237165 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:04.237168 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:04.237170 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:49:04.237217 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:49:04.237294 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260619 02:49:04.237979 7416 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42443
I20260619 02:49:04.238054 7416 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:04.238221 7416 heartbeater.cc:511] Master 127.19.113.126:42443 requested a full tablet report, sending...
I20260619 02:49:04.238556 7096 ts_manager.cc:194] Registered new tserver with Master: f982821780fe4caeb96f887ae8631cb3 (127.19.113.66:44363)
I20260619 02:49:04.238947 7096 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.66:49827
I20260619 02:49:04.241987 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 7288
I20260619 02:49:04.242077 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/wal/instance
I20260619 02:49:04.242290 19909 external_mini_cluster.cc:1428] Setting key 66d6fa693ae265af6d4303d1984ebe82
I20260619 02:49:04.243152 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.113.67:0
--local_ip_for_outbound_sockets=127.19.113.67
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42443
--builtin_ntp_servers=127.19.113.84:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:49:04.320461 7419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:04.320658 7419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:04.320688 7419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:04.321823 7419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:04.321863 7419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:04.321897 7419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.113.67
I20260619 02:49:04.323309 7419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.113.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.113.67
--webserver_port=0
--tserver_master_addrs=127.19.113.126:42443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.113.67
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:04.323526 7419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:04.323796 7419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:04.328444 7427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:04.328434 7425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:04.328482 7424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:04.328763 7419 server_base.cc:1034] running on GCE node
I20260619 02:49:04.328933 7419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:04.329138 7419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:04.330279 7419 hybrid_clock.cc:648] HybridClock initialized: now 1781837344330268 us; error 29 us; skew 500 ppm
I20260619 02:49:04.331430 7419 webserver.cc:460] Webserver started at http://127.19.113.67:35401/ using document root <none> and password file <none>
I20260619 02:49:04.331633 7419 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:04.331677 7419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:04.331795 7419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:04.332772 7419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/data/instance:
uuid: "89154928e70e42ffa39e7c90e3fa4daf"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "8c2fe3563606374019f7c5ea4cbf3d33"
server_key_iv: "f69853f4a58259022e6806e3810dcc7b"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.333073 7419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance:
uuid: "89154928e70e42ffa39e7c90e3fa4daf"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "8c2fe3563606374019f7c5ea4cbf3d33"
server_key_iv: "f69853f4a58259022e6806e3810dcc7b"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.334229 7419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:04.334904 7433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.335045 7419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:04.335122 7419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/wal
uuid: "89154928e70e42ffa39e7c90e3fa4daf"
format_stamp: "Formatted at 2026-06-19 02:49:04 on dist-test-slave-zqpb"
server_key: "8c2fe3563606374019f7c5ea4cbf3d33"
server_key_iv: "f69853f4a58259022e6806e3810dcc7b"
server_key_version: "encryptionkey@0"
I20260619 02:49:04.335183 7419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:04.346069 7419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:04.346330 7419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:04.346429 7419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:04.346585 7419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:49:04.346909 7419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:49:04.346946 7419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.346966 7419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:49:04.346980 7419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:04.352190 7419 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.67:46841
I20260619 02:49:04.352255 7546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.67:46841 every 8 connection(s)
I20260619 02:49:04.352506 7419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/data/info.pb
I20260619 02:49:04.357343 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 7419
I20260619 02:49:04.357434 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/wal/instance
I20260619 02:49:04.357475 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.353207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:52261 (local address 127.19.113.126:42443)
0619 02:49:04.353256 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.353258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.353468 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:49:04.353968 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.353970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.353982 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.354032 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:04.354669 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.355225 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.356186 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.356376 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.356431 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.357218 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:04.357220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:04.357220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:04.357225 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:04.357241 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:04.357245 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:04.357286 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.357362 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.357394 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:49:04.357654 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.353131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:42443 (local address 127.19.113.67:52261)
0619 02:49:04.353325 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.353387 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:04.353870 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:04.354106 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:04.354110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:04.354120 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:04.354581 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.354585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.355312 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.355313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.356081 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.356084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.356442 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.356443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.356574 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.357074 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:49:04.357075 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:04.357119 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:49:04.357142 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:04.357292 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:04.357297 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:04.357300 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:04.357356 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:49:04.357456 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260619 02:49:04.357669 19909 external_mini_cluster.cc:1428] Setting key a605c97c1c2c1d6a33ddefc066951719
I20260619 02:49:04.358083 7547 heartbeater.cc:347] Connected to a master server at 127.19.113.126:42443
I20260619 02:49:04.358176 7547 heartbeater.cc:464] Registering TS with master...
I20260619 02:49:04.358357 7547 heartbeater.cc:511] Master 127.19.113.126:42443 requested a full tablet report, sending...
I20260619 02:49:04.358682 7097 ts_manager.cc:194] Registered new tserver with Master: 89154928e70e42ffa39e7c90e3fa4daf (127.19.113.67:46841)
I20260619 02:49:04.359088 7097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.113.67:52261
I20260619 02:49:04.369313 19909 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:49:04.373101 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.370661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.19.113.126:42443)
0619 02:49:04.370709 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.370710 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.370810 (+ 100us) server_negotiation.cc:407] Connection header received
0619 02:49:04.370852 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.370853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.370861 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.370933 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:04.371076 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.371529 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.372223 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.372391 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.372435 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.372929 (+ 494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:04.372930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:04.372930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:04.372934 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:04.372947 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:04.372951 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:04.372986 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.373022 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.373044 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:49:04.374509 7097 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:49:04.376104 7097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:49:04.382054 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.378333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.67:46841 (local address 127.0.0.1:33904)
0619 02:49:04.378855 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.378885 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:04.378933 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:04.379143 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:04.379143 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:04.379146 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:04.379184 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.379186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.380202 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.380203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.380921 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.380923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.381778 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.381852 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.381867 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:49:04.381928 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:49:04.381981 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":51,"threads_started":1}
I20260619 02:49:04.382158 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.378185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:44363 (local address 127.0.0.1:39942)
0619 02:49:04.378472 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.378482 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:04.378525 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:04.378768 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:04.378769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:04.378771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:04.378806 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.378807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.379785 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.379786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.381271 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.381272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.381925 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.381989 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.381993 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:49:04.382046 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:49:04.382085 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":30,"thread_start_us":70,"threads_started":1}
I20260619 02:49:04.382337 7556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.378080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.19.113.65:34145)
0619 02:49:04.378301 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.378306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.378315 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:49:04.378350 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.378352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.378366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.378437 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:04.378621 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.379774 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381130 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.381801 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381856 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.381930 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.382091 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.382277 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":111,"threads_started":1}
I20260619 02:49:04.382365 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.378302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.19.113.66:44363)
0619 02:49:04.378626 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.378630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.378639 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:49:04.378674 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.378676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.378689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.378766 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:04.378914 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.379714 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381334 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.381850 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381920 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.382006 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.382062 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.382100 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":78,"threads_started":1}
I20260619 02:49:04.382388 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.378429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.19.113.67:46841)
0619 02:49:04.378562 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.378566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.378991 (+ 425us) server_negotiation.cc:407] Connection header received
0619 02:49:04.379031 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.379033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.379047 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.379112 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:04.379239 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.380135 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381005 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.381697 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381769 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.381858 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.382273 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.382314 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260619 02:49:04.382550 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.377990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:34145 (local address 127.0.0.1:53822)
0619 02:49:04.378108 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.378147 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:04.378211 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:04.378448 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:04.378450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:04.378454 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:04.378518 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.378520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.379841 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.379841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.381054 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.381057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.381960 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.381960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.382009 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.382012 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:04.382437 (+ 425us) client_negotiation.cc:240] Negotiation successful
0619 02:49:04.382486 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":49,"thread_start_us":67,"threads_started":1}
I20260619 02:49:04.383034 7350 tablet_service.cc:1467] Processing CreateTablet for tablet e0a284f01f2c4ca49d66f2b719034e6d (DEFAULT_TABLE table=test-table [id=ec9c76032ce7493483b51bd301b63d1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:04.383106 7481 tablet_service.cc:1467] Processing CreateTablet for tablet e0a284f01f2c4ca49d66f2b719034e6d (DEFAULT_TABLE table=test-table [id=ec9c76032ce7493483b51bd301b63d1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:04.383140 7219 tablet_service.cc:1467] Processing CreateTablet for tablet e0a284f01f2c4ca49d66f2b719034e6d (DEFAULT_TABLE table=test-table [id=ec9c76032ce7493483b51bd301b63d1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:49:04.383406 7481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a284f01f2c4ca49d66f2b719034e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:04.383410 7350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a284f01f2c4ca49d66f2b719034e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:04.383409 7219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a284f01f2c4ca49d66f2b719034e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:04.385859 7561 tablet_bootstrap.cc:492] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Bootstrap starting.
I20260619 02:49:04.386385 7561 tablet_bootstrap.cc:654] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:04.386539 7563 tablet_bootstrap.cc:492] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Bootstrap starting.
I20260619 02:49:04.386705 7561 log.cc:824] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:04.387005 7562 tablet_bootstrap.cc:492] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75: Bootstrap starting.
I20260619 02:49:04.387271 7563 tablet_bootstrap.cc:654] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:04.387447 7561 tablet_bootstrap.cc:492] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: No bootstrap required, opened a new log
I20260619 02:49:04.387513 7561 ts_tablet_manager.cc:1397] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:49:04.387567 7562 tablet_bootstrap.cc:654] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:04.387562 7563 log.cc:824] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:04.387861 7562 log.cc:824] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:04.388331 7563 tablet_bootstrap.cc:492] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: No bootstrap required, opened a new log
I20260619 02:49:04.388442 7563 ts_tablet_manager.cc:1397] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:49:04.388480 7562 tablet_bootstrap.cc:492] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75: No bootstrap required, opened a new log
I20260619 02:49:04.388525 7562 ts_tablet_manager.cc:1397] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:49:04.389951 7561 raft_consensus.cc:348] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.390113 7561 raft_consensus.cc:374] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:04.390147 7561 raft_consensus.cc:729] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89154928e70e42ffa39e7c90e3fa4daf, State: Initialized, Role: FOLLOWER
I20260619 02:49:04.390265 7561 consensus_queue.cc:260] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.390498 7561 ts_tablet_manager.cc:1428] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:49:04.390431 7562 raft_consensus.cc:348] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.390547 7562 raft_consensus.cc:374] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:04.390583 7562 raft_consensus.cc:729] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c59da9220894887b2d0e709b280df75, State: Initialized, Role: FOLLOWER
I20260619 02:49:04.390630 7547 heartbeater.cc:503] Master 127.19.113.126:42443 was elected leader, sending a full tablet report...
I20260619 02:49:04.390722 7562 consensus_queue.cc:260] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.390980 7562 ts_tablet_manager.cc:1428] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:49:04.391100 7285 heartbeater.cc:503] Master 127.19.113.126:42443 was elected leader, sending a full tablet report...
I20260619 02:49:04.391464 7563 raft_consensus.cc:348] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.391723 7563 raft_consensus.cc:374] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:04.391762 7563 raft_consensus.cc:729] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f982821780fe4caeb96f887ae8631cb3, State: Initialized, Role: FOLLOWER
I20260619 02:49:04.391875 7563 consensus_queue.cc:260] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.392120 7563 ts_tablet_manager.cc:1428] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:49:04.392148 7416 heartbeater.cc:503] Master 127.19.113.126:42443 was elected leader, sending a full tablet report...
W20260619 02:49:04.483023 7417 tablet.cc:2369] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:49:04.603026 7548 tablet.cc:2369] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:49:04.619971 7286 tablet.cc:2369] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:49:04.647039 7567 raft_consensus.cc:482] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:49:04.647181 7567 raft_consensus.cc:504] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.647521 7567 leader_election.cc:288] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c59da9220894887b2d0e709b280df75 (127.19.113.65:34145), f982821780fe4caeb96f887ae8631cb3 (127.19.113.66:44363)
I20260619 02:49:04.650135 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.647720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:34145 (local address 127.19.113.67:48521)
0619 02:49:04.647771 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.647782 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:04.647839 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:04.647954 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:04.647955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:04.647957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:04.648039 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.648041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.648651 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.648652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.649387 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.649389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.649907 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.649908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.649975 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.649978 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:49:04.650020 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:49:04.650060 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:49:04.650135 7556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.647785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:48521 (local address 127.19.113.65:34145)
0619 02:49:04.647843 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.647844 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.647854 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:49:04.647887 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.647888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.647898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.647951 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:04.648105 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.648570 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.649491 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.649844 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.649904 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.649912 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.650033 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.650067 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:49:04.650424 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.648081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.113.67:50739 (local address 127.19.113.66:44363)
0619 02:49:04.648135 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.648136 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.648148 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:49:04.648172 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.648173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.648180 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.648222 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:49:04.648365 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.648816 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.649724 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.650080 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.650138 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.650145 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.650318 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.650364 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:49:04.650441 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.647828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.66:44363 (local address 127.19.113.67:50739)
0619 02:49:04.647972 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.647982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:04.648048 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:04.648236 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:04.648237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:04.648238 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:49:04.648305 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.648307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.648887 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.648888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.649635 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:04.649637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.650155 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.650156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:04.650218 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.650222 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:49:04.650302 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:49:04.650347 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260619 02:49:04.650529 7239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a284f01f2c4ca49d66f2b719034e6d" candidate_uuid: "89154928e70e42ffa39e7c90e3fa4daf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c59da9220894887b2d0e709b280df75" is_pre_election: true
I20260619 02:49:04.650682 7239 raft_consensus.cc:2454] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89154928e70e42ffa39e7c90e3fa4daf in term 0.
I20260619 02:49:04.650758 7370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a284f01f2c4ca49d66f2b719034e6d" candidate_uuid: "89154928e70e42ffa39e7c90e3fa4daf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f982821780fe4caeb96f887ae8631cb3" is_pre_election: true
I20260619 02:49:04.650899 7370 raft_consensus.cc:2454] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89154928e70e42ffa39e7c90e3fa4daf in term 0.
I20260619 02:49:04.650938 7435 leader_election.cc:302] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c59da9220894887b2d0e709b280df75, 89154928e70e42ffa39e7c90e3fa4daf; no voters:
I20260619 02:49:04.651108 7567 raft_consensus.cc:2781] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:49:04.651170 7567 raft_consensus.cc:482] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:49:04.651183 7567 raft_consensus.cc:3037] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:04.651821 7567 raft_consensus.cc:504] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.651921 7567 leader_election.cc:288] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [CANDIDATE]: Term 1 election: Requested vote from peers 2c59da9220894887b2d0e709b280df75 (127.19.113.65:34145), f982821780fe4caeb96f887ae8631cb3 (127.19.113.66:44363)
I20260619 02:49:04.652079 7239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a284f01f2c4ca49d66f2b719034e6d" candidate_uuid: "89154928e70e42ffa39e7c90e3fa4daf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c59da9220894887b2d0e709b280df75"
I20260619 02:49:04.652143 7370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a284f01f2c4ca49d66f2b719034e6d" candidate_uuid: "89154928e70e42ffa39e7c90e3fa4daf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f982821780fe4caeb96f887ae8631cb3"
I20260619 02:49:04.652160 7239 raft_consensus.cc:3037] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:04.652208 7370 raft_consensus.cc:3037] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:04.653012 7239 raft_consensus.cc:2454] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89154928e70e42ffa39e7c90e3fa4daf in term 1.
I20260619 02:49:04.653012 7370 raft_consensus.cc:2454] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89154928e70e42ffa39e7c90e3fa4daf in term 1.
I20260619 02:49:04.653224 7435 leader_election.cc:302] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c59da9220894887b2d0e709b280df75, 89154928e70e42ffa39e7c90e3fa4daf; no voters:
I20260619 02:49:04.653306 7567 raft_consensus.cc:2781] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:04.653474 7567 raft_consensus.cc:686] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 1 LEADER]: Becoming Leader. State: Replica: 89154928e70e42ffa39e7c90e3fa4daf, State: Running, Role: LEADER
I20260619 02:49:04.653584 7567 consensus_queue.cc:237] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } }
I20260619 02:49:04.654363 7097 catalog_manager.cc:5495] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf reported cstate change: term changed from 0 to 1, leader changed from <none> to 89154928e70e42ffa39e7c90e3fa4daf (127.19.113.67). New cstate: current_term: 1 leader_uuid: "89154928e70e42ffa39e7c90e3fa4daf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89154928e70e42ffa39e7c90e3fa4daf" member_type: VOTER last_known_addr { host: "127.19.113.67" port: 46841 } health_report { overall_health: HEALTHY } } }
I20260619 02:49:04.664885 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.662787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33916 (local address 127.19.113.67:46841)
0619 02:49:04.662834 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:04.662835 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:04.662842 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:49:04.662915 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:04.662916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:04.662934 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:04.663005 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:04.663122 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.663559 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.664239 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:04.664414 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:04.664462 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:04.664601 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:04.664696 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:04.664786 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:04.664799 (+ 13us) server_negotiation.cc:299] Negotiation successful
0619 02:49:04.664817 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:49:04.666507 7239 raft_consensus.cc:1261] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 1 FOLLOWER]: Refusing update from remote peer 89154928e70e42ffa39e7c90e3fa4daf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:04.666507 7370 raft_consensus.cc:1261] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 1 FOLLOWER]: Refusing update from remote peer 89154928e70e42ffa39e7c90e3fa4daf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:04.666812 7567 consensus_queue.cc:1035] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f982821780fe4caeb96f887ae8631cb3" member_type: VOTER last_known_addr { host: "127.19.113.66" port: 44363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:04.666971 7571 consensus_queue.cc:1035] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c59da9220894887b2d0e709b280df75" member_type: VOTER last_known_addr { host: "127.19.113.65" port: 34145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:04.669652 7576 mvcc.cc:205] Tried to move back new op lower bound from 7298405763751673856 to 7298405763701624832. Current Snapshot: MvccSnapshot[applied={T|T < 7298405763751673856 or (T in {7298405763751673856})}]
I20260619 02:49:04.670080 7575 mvcc.cc:205] Tried to move back new op lower bound from 7298405763751673856 to 7298405763701624832. Current Snapshot: MvccSnapshot[applied={T|T < 7298405763751673856}]
I20260619 02:49:04.671200 7097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:49:04.671288 7097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:49:04.671981 7097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6031adcf1b91462789184e0ae562d00c: Sending DeleteTablet for 3 replicas of tablet e0a284f01f2c4ca49d66f2b719034e6d
I20260619 02:49:04.672266 7219 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a284f01f2c4ca49d66f2b719034e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:04 UTC) from {username='slave'} at 127.0.0.1:53822
I20260619 02:49:04.672341 7350 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a284f01f2c4ca49d66f2b719034e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:04 UTC) from {username='slave'} at 127.0.0.1:39942
I20260619 02:49:04.672363 7481 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a284f01f2c4ca49d66f2b719034e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:49:04 UTC) from {username='slave'} at 127.0.0.1:33904
I20260619 02:49:04.672560 7583 tablet_replica.cc:330] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75: stopping tablet replica
I20260619 02:49:04.672662 7583 raft_consensus.cc:2229] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:49:04.672755 7583 raft_consensus.cc:2258] T e0a284f01f2c4ca49d66f2b719034e6d P 2c59da9220894887b2d0e709b280df75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:04.672756 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 7157
I20260619 02:49:04.672888 7585 tablet_replica.cc:330] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: stopping tablet replica
I20260619 02:49:04.672961 7585 raft_consensus.cc:2229] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:49:04.673027 7585 raft_consensus.cc:2258] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:04.673319 7584 tablet_replica.cc:330] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: stopping tablet replica
I20260619 02:49:04.673399 7584 raft_consensus.cc:2229] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:49:04.673477 7584 raft_consensus.cc:2258] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:49:04.673622 7585 ts_tablet_manager.cc:1905] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:49:04.673868 7584 ts_tablet_manager.cc:1905] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:49:04.675184 7585 ts_tablet_manager.cc:1918] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:49:04.675246 7585 log.cc:1196] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-1/wal/wals/e0a284f01f2c4ca49d66f2b719034e6d
I20260619 02:49:04.675313 7584 ts_tablet_manager.cc:1918] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:49:04.675362 7584 log.cc:1196] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Deleting WAL directory at /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837111403550-19909-0/minicluster-data/ts-2/wal/wals/e0a284f01f2c4ca49d66f2b719034e6d
I20260619 02:49:04.675469 7585 ts_tablet_manager.cc:1939] T e0a284f01f2c4ca49d66f2b719034e6d P f982821780fe4caeb96f887ae8631cb3: Deleting consensus metadata
I20260619 02:49:04.675562 7584 ts_tablet_manager.cc:1939] T e0a284f01f2c4ca49d66f2b719034e6d P 89154928e70e42ffa39e7c90e3fa4daf: Deleting consensus metadata
I20260619 02:49:04.675832 7082 catalog_manager.cc:4841] TS f982821780fe4caeb96f887ae8631cb3 (127.19.113.66:44363): tablet e0a284f01f2c4ca49d66f2b719034e6d (table test-table [id=ec9c76032ce7493483b51bd301b63d1a]) successfully deleted
I20260619 02:49:04.675911 7082 catalog_manager.cc:4841] TS 89154928e70e42ffa39e7c90e3fa4daf (127.19.113.67:46841): tablet e0a284f01f2c4ca49d66f2b719034e6d (table test-table [id=ec9c76032ce7493483b51bd301b63d1a]) successfully deleted
W20260619 02:49:04.680327 7082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.113.65:34145 (error 108)
I20260619 02:49:04.680533 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 7288
I20260619 02:49:04.681149 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:04.680649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.65:34145 (local address 127.0.0.1:53828)
0619 02:49:04.681064 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:04.681114 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.65:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391}
W20260619 02:49:04.681279 7082 catalog_manager.cc:4584] TS 2c59da9220894887b2d0e709b280df75 (127.19.113.65:34145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e0a284f01f2c4ca49d66f2b719034e6d: Network error: Client connection negotiation failed: client connection to 127.19.113.65:34145: connect: Connection refused (error 111)
I20260619 02:49:04.685451 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 7419
I20260619 02:49:04.689168 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 7066
2026-06-19T02:49:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (892 ms)
[----------] 6 tests from EncryptionPolicyTest (7999 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:04 dist-test-slave-zqpb krb5kdc[7589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:04 dist-test-slave-zqpb krb5kdc[7589](info): set up 2 sockets
Jun 19 02:49:04 dist-test-slave-zqpb krb5kdc[7589](info): commencing operation
krb5kdc: starting...
W20260619 02:49:05.634662 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:05 dist-test-slave-zqpb krb5kdc[7589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:05.742895 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:05.820973 7602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:05.821183 7602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:05.821205 7602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:05.822278 7602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:05.822311 7602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:05.822324 7602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:05.822335 7602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:05.822350 7602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:05.823807 7602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:05.823994 7602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:05.824167 7602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:05.828627 7610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:05.828615 7607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:05.828644 7602 server_base.cc:1034] running on GCE node
W20260619 02:49:05.828600 7608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:05.829048 7602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:05.829332 7602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:05.830470 7602 hybrid_clock.cc:648] HybridClock initialized: now 1781837345830460 us; error 29 us; skew 500 ppm
Jun 19 02:49:05 dist-test-slave-zqpb krb5kdc[7589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837345, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:05.832723 7602 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:05.833132 7602 webserver.cc:460] Webserver started at http://127.0.0.1:40023/ using document root <none> and password file <none>
I20260619 02:49:05.833333 7602 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:05.833379 7602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:05.833498 7602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:05.834457 7602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "e03ab6fd55a24690b1eae6df0e4bb990"
format_stamp: "Formatted at 2026-06-19 02:49:05 on dist-test-slave-zqpb"
server_key: "d61766916ed9872b1ddc200ef1e7dc94"
server_key_iv: "c3826e154f02d32419191ed787d87438"
server_key_version: "encryptionkey@0"
I20260619 02:49:05.834774 7602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "e03ab6fd55a24690b1eae6df0e4bb990"
format_stamp: "Formatted at 2026-06-19 02:49:05 on dist-test-slave-zqpb"
server_key: "d61766916ed9872b1ddc200ef1e7dc94"
server_key_iv: "c3826e154f02d32419191ed787d87438"
server_key_version: "encryptionkey@0"
I20260619 02:49:05.836084 7602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:49:05.836771 7617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:05.836916 7602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:49:05.837003 7602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "e03ab6fd55a24690b1eae6df0e4bb990"
format_stamp: "Formatted at 2026-06-19 02:49:05 on dist-test-slave-zqpb"
server_key: "d61766916ed9872b1ddc200ef1e7dc94"
server_key_iv: "c3826e154f02d32419191ed787d87438"
server_key_version: "encryptionkey@0"
I20260619 02:49:05.837074 7602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:05.856645 7602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:05.858369 7602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:05.858506 7602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:05.862316 7602 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32823
I20260619 02:49:05.862354 7669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32823 every 8 connection(s)
I20260619 02:49:05.862673 7602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:05.863476 7670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:05.865635 7670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990: Bootstrap starting.
I20260619 02:49:05.866159 7670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:05.866401 7670 log.cc:824] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:05.867041 7670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990: No bootstrap required, opened a new log
I20260619 02:49:05.867264 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 7602
I20260619 02:49:05.867365 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:05.867516 19909 external_mini_cluster.cc:1428] Setting key fc3d4cbb44f3ad0137f60a24dbcdf6be
I20260619 02:49:05.869148 7670 raft_consensus.cc:348] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03ab6fd55a24690b1eae6df0e4bb990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } }
I20260619 02:49:05.869279 7670 raft_consensus.cc:374] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:05.869300 7670 raft_consensus.cc:729] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e03ab6fd55a24690b1eae6df0e4bb990, State: Initialized, Role: FOLLOWER
I20260619 02:49:05.869402 7670 consensus_queue.cc:260] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03ab6fd55a24690b1eae6df0e4bb990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } }
I20260619 02:49:05.869472 7670 raft_consensus.cc:388] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:05.869495 7670 raft_consensus.cc:482] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:05.869539 7670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:05.870468 7670 raft_consensus.cc:504] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03ab6fd55a24690b1eae6df0e4bb990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } }
I20260619 02:49:05.870569 7670 leader_election.cc:302] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e03ab6fd55a24690b1eae6df0e4bb990; no voters:
I20260619 02:49:05.870705 7670 leader_election.cc:288] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:05.870733 7675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:05.870886 7675 raft_consensus.cc:686] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [term 1 LEADER]: Becoming Leader. State: Replica: e03ab6fd55a24690b1eae6df0e4bb990, State: Running, Role: LEADER
I20260619 02:49:05.870940 7670 sys_catalog.cc:564] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:05.870975 7675 consensus_queue.cc:237] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03ab6fd55a24690b1eae6df0e4bb990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } }
I20260619 02:49:05.871306 7677 sys_catalog.cc:455] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e03ab6fd55a24690b1eae6df0e4bb990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03ab6fd55a24690b1eae6df0e4bb990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } } }
I20260619 02:49:05.871392 7677 sys_catalog.cc:458] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:05.871296 7675 sys_catalog.cc:455] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e03ab6fd55a24690b1eae6df0e4bb990. Latest consensus state: current_term: 1 leader_uuid: "e03ab6fd55a24690b1eae6df0e4bb990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03ab6fd55a24690b1eae6df0e4bb990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } } }
I20260619 02:49:05.871554 7675 sys_catalog.cc:458] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:05.871695 7682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:05.872344 7682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:05.874164 7682 catalog_manager.cc:1347] Generated new cluster ID: ca5dad406fe64bf0bca44c79c315ddc8
I20260619 02:49:05.874215 7682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:49:05 dist-test-slave-zqpb krb5kdc[7589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:49:05.877987 7674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:05.868017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47601 (local address 127.0.0.1:32823)
0619 02:49:05.868246 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:05.868256 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:05.868289 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:49:05.868744 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:05.868753 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:05.868812 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:05.868959 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:05.869598 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:05.870938 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:05.872053 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:05.872663 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:05.872768 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:05.875156 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:05.875160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:05.875160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:05.875171 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:05.876605 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:05.876948 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:05.876951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:05.876951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:05.877028 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:05.877224 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:05.877225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:05.877225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:05.877448 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:05.877596 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:05.877722 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:49:05.877814 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":126,"threads_started":1}
I20260619 02:49:05.878854 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:05.884722 7682 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:05.885382 7682 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:05.888480 7674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:05.880104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44601 (local address 127.0.0.1:32823)
0619 02:49:05.880143 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:05.880144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:05.883028 (+ 2884us) server_negotiation.cc:407] Connection header received
0619 02:49:05.883081 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:05.883082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:05.883122 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:05.883167 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:05.883620 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:05.884103 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:05.884938 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:05.885155 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:05.885207 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:05.886261 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:05.886262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:05.886262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:05.886268 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:05.887468 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:05.887789 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:05.887790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:05.887790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:05.887840 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:05.888035 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:05.888035 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:05.888036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:05.888139 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:05.888227 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:05.888344 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:49:05.888399 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":21}
I20260619 02:49:05.897385 7682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e03ab6fd55a24690b1eae6df0e4bb990: Generated new TSK 0
I20260619 02:49:05.897547 7682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:05.902518 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 7602
W20260619 02:49:05.906489 7587 connection.cc:535] client connection to 127.0.0.1:32823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32823): Connection reset by peer (error 104)
2026-06-19T02:49:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1211 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:05 dist-test-slave-zqpb krb5kdc[7701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:05 dist-test-slave-zqpb krb5kdc[7701](info): set up 2 sockets
Jun 19 02:49:05 dist-test-slave-zqpb krb5kdc[7701](info): commencing operation
krb5kdc: starting...
W20260619 02:49:06.846693 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:06 dist-test-slave-zqpb krb5kdc[7701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:06.957486 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42587
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:07.035974 7714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:07.036185 7714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:07.036206 7714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:07.037320 7714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:07.037355 7714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:07.037369 7714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:07.037379 7714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:07.037389 7714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:07.038782 7714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42587
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:07.039098 7714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:07.039310 7714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:07.043759 7722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:07.043759 7719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:07.043751 7720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:07.044219 7714 server_base.cc:1034] running on GCE node
I20260619 02:49:07.044435 7714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:07.044699 7714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:07.045845 7714 hybrid_clock.cc:648] HybridClock initialized: now 1781837347045832 us; error 25 us; skew 500 ppm
Jun 19 02:49:07 dist-test-slave-zqpb krb5kdc[7701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837347, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:07.048264 7714 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:07.048699 7714 webserver.cc:460] Webserver started at http://127.0.0.1:41665/ using document root <none> and password file <none>
I20260619 02:49:07.048913 7714 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:07.048960 7714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:07.049088 7714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:07.050109 7714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "34f1e5df326c4cfea33bd6273299a887"
format_stamp: "Formatted at 2026-06-19 02:49:07 on dist-test-slave-zqpb"
server_key: "2db4138b201729ade30da8e8ef216d1e"
server_key_iv: "31d77eb4d2e88b53657df8e36a5c42ab"
server_key_version: "encryptionkey@0"
I20260619 02:49:07.050433 7714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "34f1e5df326c4cfea33bd6273299a887"
format_stamp: "Formatted at 2026-06-19 02:49:07 on dist-test-slave-zqpb"
server_key: "2db4138b201729ade30da8e8ef216d1e"
server_key_iv: "31d77eb4d2e88b53657df8e36a5c42ab"
server_key_version: "encryptionkey@0"
I20260619 02:49:07.051752 7714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:49:07.052515 7729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:07.052673 7714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:49:07.052747 7714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "34f1e5df326c4cfea33bd6273299a887"
format_stamp: "Formatted at 2026-06-19 02:49:07 on dist-test-slave-zqpb"
server_key: "2db4138b201729ade30da8e8ef216d1e"
server_key_iv: "31d77eb4d2e88b53657df8e36a5c42ab"
server_key_version: "encryptionkey@0"
I20260619 02:49:07.052809 7714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:07.063191 7714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:07.064819 7714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:07.064942 7714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:07.068665 7714 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42587
I20260619 02:49:07.068696 7781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42587 every 8 connection(s)
I20260619 02:49:07.069008 7714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:07.069815 7782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:07.071907 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 7714
I20260619 02:49:07.072014 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:07.072214 19909 external_mini_cluster.cc:1428] Setting key 079e39a10a3d0387c92782c2c50b4734
I20260619 02:49:07.072201 7782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887: Bootstrap starting.
I20260619 02:49:07.072849 7782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:07.073118 7782 log.cc:824] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:07.073820 7782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887: No bootstrap required, opened a new log
I20260619 02:49:07.075860 7782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f1e5df326c4cfea33bd6273299a887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } }
I20260619 02:49:07.076006 7782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:07.076028 7782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f1e5df326c4cfea33bd6273299a887, State: Initialized, Role: FOLLOWER
I20260619 02:49:07.076156 7782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f1e5df326c4cfea33bd6273299a887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } }
I20260619 02:49:07.076222 7782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:07.076243 7782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:07.076287 7782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:07.076941 7782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f1e5df326c4cfea33bd6273299a887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } }
I20260619 02:49:07.077049 7782 leader_election.cc:302] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34f1e5df326c4cfea33bd6273299a887; no voters:
I20260619 02:49:07.077203 7782 leader_election.cc:288] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:07.077229 7787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:07.077374 7787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [term 1 LEADER]: Becoming Leader. State: Replica: 34f1e5df326c4cfea33bd6273299a887, State: Running, Role: LEADER
I20260619 02:49:07.077445 7782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:07.077459 7787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f1e5df326c4cfea33bd6273299a887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } }
I20260619 02:49:07.077787 7788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f1e5df326c4cfea33bd6273299a887. Latest consensus state: current_term: 1 leader_uuid: "34f1e5df326c4cfea33bd6273299a887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f1e5df326c4cfea33bd6273299a887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } } }
I20260619 02:49:07.077875 7788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:07.077833 7789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34f1e5df326c4cfea33bd6273299a887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f1e5df326c4cfea33bd6273299a887" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42587 } } }
I20260619 02:49:07.078037 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:07.072699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39281 (local address 127.0.0.1:42587)
0619 02:49:07.072859 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:07.072867 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:07.072894 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:49:07.073405 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:07.073414 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:07.073475 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:07.073633 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:07.073842 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:07.074763 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:07.075886 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:07.076381 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:07.076463 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:07.077113 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:07.077117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:07.077117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:07.077323 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:07.077361 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:07.077376 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:07.077458 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:07.077750 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 02:49:07.077780 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260619 02:49:07.078043 7789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:07.078662 7794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:07.078976 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:07.079327 7794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:07.080842 7794 catalog_manager.cc:1347] Generated new cluster ID: 3588e89731df48338b3baa65b9adc057
I20260619 02:49:07.080896 7794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:07.083825 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:07.080448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47237 (local address 127.0.0.1:42587)
0619 02:49:07.080484 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:07.080486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:07.080560 (+ 74us) server_negotiation.cc:407] Connection header received
0619 02:49:07.080634 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:07.080635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:07.080644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:07.080695 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:07.080837 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:07.081581 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:07.082566 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:07.082909 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:07.082988 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:07.083558 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:07.083559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:07.083559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:07.083564 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:07.083579 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:07.083583 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:07.083647 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:07.083738 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:49:07.083750 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:49:07.091684 7794 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:07.092178 7794 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:07.094372 7794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34f1e5df326c4cfea33bd6273299a887: Generated new TSK 0
I20260619 02:49:07.094520 7794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:07.100580 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 7714
W20260619 02:49:07.104313 7699 connection.cc:535] client connection to 127.0.0.1:42587 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42587): Connection reset by peer (error 104)
2026-06-19T02:49:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1197 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:07 dist-test-slave-zqpb krb5kdc[7813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:07 dist-test-slave-zqpb krb5kdc[7813](info): set up 2 sockets
Jun 19 02:49:07 dist-test-slave-zqpb krb5kdc[7813](info): commencing operation
krb5kdc: starting...
W20260619 02:49:08.054741 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:08 dist-test-slave-zqpb krb5kdc[7813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:08.166704 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:08.246186 7826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:08.246392 7826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:08.246413 7826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:08.247610 7826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:08.247650 7826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:08.247663 7826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:08.247676 7826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:08.247701 7826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:08.249178 7826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38305
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:08.249393 7826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:08.249600 7826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:08.254128 7834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:08.254096 7831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:08.254134 7826 server_base.cc:1034] running on GCE node
W20260619 02:49:08.254101 7832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:08.254541 7826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:08.254817 7826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:08.255970 7826 hybrid_clock.cc:648] HybridClock initialized: now 1781837348255965 us; error 39 us; skew 500 ppm
Jun 19 02:49:08 dist-test-slave-zqpb krb5kdc[7813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837348, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:08.258309 7826 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:08.258713 7826 webserver.cc:460] Webserver started at http://127.0.0.1:34465/ using document root <none> and password file <none>
I20260619 02:49:08.258936 7826 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:08.258984 7826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:08.259109 7826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:08.260134 7826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "fdb57d9b416747ebb738a7bbedb53676"
format_stamp: "Formatted at 2026-06-19 02:49:08 on dist-test-slave-zqpb"
server_key: "4c375095a0461e5865640e7ce30af0a2"
server_key_iv: "fc7a51453671b14b56cbd361e2cc2e5f"
server_key_version: "encryptionkey@0"
I20260619 02:49:08.260456 7826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "fdb57d9b416747ebb738a7bbedb53676"
format_stamp: "Formatted at 2026-06-19 02:49:08 on dist-test-slave-zqpb"
server_key: "4c375095a0461e5865640e7ce30af0a2"
server_key_iv: "fc7a51453671b14b56cbd361e2cc2e5f"
server_key_version: "encryptionkey@0"
I20260619 02:49:08.261652 7826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:08.262284 7841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:08.262424 7826 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:08.262504 7826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "fdb57d9b416747ebb738a7bbedb53676"
format_stamp: "Formatted at 2026-06-19 02:49:08 on dist-test-slave-zqpb"
server_key: "4c375095a0461e5865640e7ce30af0a2"
server_key_iv: "fc7a51453671b14b56cbd361e2cc2e5f"
server_key_version: "encryptionkey@0"
I20260619 02:49:08.262563 7826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:08.265820 7826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:08.267822 7826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:08.267948 7826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:08.271525 7826 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38305
I20260619 02:49:08.271556 7893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38305 every 8 connection(s)
I20260619 02:49:08.271853 7826 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:08.272639 7894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:08.274781 7894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676: Bootstrap starting.
I20260619 02:49:08.275339 7894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:08.275579 7894 log.cc:824] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:08.276289 7894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676: No bootstrap required, opened a new log
I20260619 02:49:08.277911 7894 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb57d9b416747ebb738a7bbedb53676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260619 02:49:08.278048 7894 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:08.278069 7894 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdb57d9b416747ebb738a7bbedb53676, State: Initialized, Role: FOLLOWER
I20260619 02:49:08.278203 7894 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb57d9b416747ebb738a7bbedb53676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260619 02:49:08.278262 7894 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:08.278282 7894 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:08.278324 7894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:08.278846 7894 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb57d9b416747ebb738a7bbedb53676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260619 02:49:08.278965 7894 leader_election.cc:302] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdb57d9b416747ebb738a7bbedb53676; no voters:
I20260619 02:49:08.279124 7894 leader_election.cc:288] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:08.279156 7897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:08.279304 7897 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [term 1 LEADER]: Becoming Leader. State: Replica: fdb57d9b416747ebb738a7bbedb53676, State: Running, Role: LEADER
I20260619 02:49:08.279356 7894 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:08.279392 7897 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb57d9b416747ebb738a7bbedb53676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260619 02:49:08.279760 7899 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdb57d9b416747ebb738a7bbedb53676. Latest consensus state: current_term: 1 leader_uuid: "fdb57d9b416747ebb738a7bbedb53676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb57d9b416747ebb738a7bbedb53676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } } }
I20260619 02:49:08.279839 7899 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:08.279883 7898 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdb57d9b416747ebb738a7bbedb53676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb57d9b416747ebb738a7bbedb53676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } } }
I20260619 02:49:08.279942 7898 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:08.280043 7906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:08.280521 7906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:08.281394 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 7826
I20260619 02:49:08.281487 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:08.281663 19909 external_mini_cluster.cc:1428] Setting key 661d7abf8a6c34724f4e2456c920da88
I20260619 02:49:08.283542 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:08.282131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37117 (local address 127.0.0.1:38305)
0619 02:49:08.282273 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:08.282282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:08.282313 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:49:08.282404 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:08.282409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:08.282433 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:08.282542 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:08.283161 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:08.283164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:08.283164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:08.283168 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:08.283201 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:08.283211 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:08.283264 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:08.283332 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:49:08.283364 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260619 02:49:08.284108 7906 catalog_manager.cc:1347] Generated new cluster ID: 0800774447a342aba8843d223b21b9e6
I20260619 02:49:08.284159 7906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:08.284447 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:08.286468 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:08.285645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60921 (local address 127.0.0.1:38305)
0619 02:49:08.285675 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:08.285677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:08.285770 (+ 93us) server_negotiation.cc:407] Connection header received
0619 02:49:08.285821 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:08.285822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:08.285827 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:08.285871 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:08.286302 (+ 431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:08.286303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:08.286303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:08.286306 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:08.286318 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:08.286322 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:08.286357 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:08.286404 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:49:08.286413 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 02:49:08.288305 7906 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:08.288445 7906 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:08.295233 7906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdb57d9b416747ebb738a7bbedb53676: Generated new TSK 0
I20260619 02:49:08.295403 7906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:08.302287 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 7826
2026-06-19T02:49:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1201 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:08 dist-test-slave-zqpb krb5kdc[7925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:08 dist-test-slave-zqpb krb5kdc[7925](info): set up 2 sockets
Jun 19 02:49:08 dist-test-slave-zqpb krb5kdc[7925](info): commencing operation
krb5kdc: starting...
W20260619 02:49:09.250681 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:09 dist-test-slave-zqpb krb5kdc[7925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:09.357757 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:09.434818 7938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:09.435051 7938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:09.435082 7938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:09.436522 7938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:09.436584 7938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:09.436606 7938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:09.436625 7938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:09.436640 7938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:09.438341 7938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:09.438598 7938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:09.438807 7938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:09.443542 7943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:09.443621 7938 server_base.cc:1034] running on GCE node
W20260619 02:49:09.443537 7944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:09.443583 7946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:09.444010 7938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:09.444312 7938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:09.445519 7938 hybrid_clock.cc:648] HybridClock initialized: now 1781837349445508 us; error 30 us; skew 500 ppm
Jun 19 02:49:09 dist-test-slave-zqpb krb5kdc[7925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837349, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:09.452808 7938 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:09.453249 7938 webserver.cc:460] Webserver started at http://127.0.0.1:40339/ using document root <none> and password file <none>
I20260619 02:49:09.453570 7938 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:09.453645 7938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:09.453759 7938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:09.454809 7938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "d84c6dc0a5634bc398f5d62f410b5dad"
format_stamp: "Formatted at 2026-06-19 02:49:09 on dist-test-slave-zqpb"
server_key: "fd75972f2a7e9a4d9aedad249d7a8bc5"
server_key_iv: "e79baabe07ef21483142c8b347bec196"
server_key_version: "encryptionkey@0"
I20260619 02:49:09.455175 7938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "d84c6dc0a5634bc398f5d62f410b5dad"
format_stamp: "Formatted at 2026-06-19 02:49:09 on dist-test-slave-zqpb"
server_key: "fd75972f2a7e9a4d9aedad249d7a8bc5"
server_key_iv: "e79baabe07ef21483142c8b347bec196"
server_key_version: "encryptionkey@0"
I20260619 02:49:09.456383 7938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:49:09.457023 7953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:09.457152 7938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:09.457232 7938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "d84c6dc0a5634bc398f5d62f410b5dad"
format_stamp: "Formatted at 2026-06-19 02:49:09 on dist-test-slave-zqpb"
server_key: "fd75972f2a7e9a4d9aedad249d7a8bc5"
server_key_iv: "e79baabe07ef21483142c8b347bec196"
server_key_version: "encryptionkey@0"
I20260619 02:49:09.457288 7938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:09.465394 7938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:09.467027 7938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:09.467228 7938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:09.471230 7938 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45575
I20260619 02:49:09.471256 8005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45575 every 8 connection(s)
I20260619 02:49:09.471590 7938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:09.471813 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 7938
I20260619 02:49:09.471935 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:09.472131 19909 external_mini_cluster.cc:1428] Setting key d75fbd050054b067b0c7870eb750a1ef
I20260619 02:49:09.472519 8006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:09.475231 8006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad: Bootstrap starting.
I20260619 02:49:09.475888 8006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:09.476176 8006 log.cc:824] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:09.476809 8006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad: No bootstrap required, opened a new log
Jun 19 02:49:09 dist-test-slave-zqpb krb5kdc[7925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:49:09.479039 8006 raft_consensus.cc:348] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84c6dc0a5634bc398f5d62f410b5dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45575 } }
I20260619 02:49:09.479172 8006 raft_consensus.cc:374] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:09.479205 8006 raft_consensus.cc:729] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84c6dc0a5634bc398f5d62f410b5dad, State: Initialized, Role: FOLLOWER
I20260619 02:49:09.479342 8006 consensus_queue.cc:260] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84c6dc0a5634bc398f5d62f410b5dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45575 } }
I20260619 02:49:09.479416 8006 raft_consensus.cc:388] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:09.479458 8006 raft_consensus.cc:482] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:09.479504 8006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:09.480034 8006 raft_consensus.cc:504] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84c6dc0a5634bc398f5d62f410b5dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45575 } }
I20260619 02:49:09.480124 8006 leader_election.cc:302] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d84c6dc0a5634bc398f5d62f410b5dad; no voters:
I20260619 02:49:09.480264 8006 leader_election.cc:288] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:09.480348 8011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:09.480506 8006 sys_catalog.cc:564] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:09.480562 8011 raft_consensus.cc:686] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [term 1 LEADER]: Becoming Leader. State: Replica: d84c6dc0a5634bc398f5d62f410b5dad, State: Running, Role: LEADER
I20260619 02:49:09.480700 8011 consensus_queue.cc:237] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84c6dc0a5634bc398f5d62f410b5dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45575 } }
I20260619 02:49:09.481060 8012 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d84c6dc0a5634bc398f5d62f410b5dad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84c6dc0a5634bc398f5d62f410b5dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45575 } } }
I20260619 02:49:09.481137 8013 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [sys.catalog]: SysCatalogTable state changed. Reason: New leader d84c6dc0a5634bc398f5d62f410b5dad. Latest consensus state: current_term: 1 leader_uuid: "d84c6dc0a5634bc398f5d62f410b5dad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84c6dc0a5634bc398f5d62f410b5dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45575 } } }
I20260619 02:49:09.481242 8012 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:09.481398 8013 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad [sys.catalog]: This master's current role is: LEADER
W20260619 02:49:09.482542 8026 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:09.482636 8026 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:09.482753 8025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:09.482985 8025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:09.483404 8009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:09.472676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40985 (local address 127.0.0.1:45575)
0619 02:49:09.472856 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:09.472866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:09.472904 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:49:09.473541 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:09.473548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:09.473595 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:09.473718 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:09.474334 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:09.475251 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:09.476320 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:09.476738 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:09.476821 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:09.479280 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:09.479284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:09.479284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:09.479294 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:09.481286 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:09.481699 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:09.481702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:09.481703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:09.481805 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:09.482181 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:09.482182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:09.482182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:09.482492 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:09.483075 (+ 583us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:09.483146 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:49:09.483264 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260619 02:49:09.484676 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:09.485548 8025 catalog_manager.cc:1347] Generated new cluster ID: a61684b0e65c4b0dba0a5259f9f3de2c
I20260619 02:49:09.485598 8025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:09.497968 8009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:09.491008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59691 (local address 127.0.0.1:45575)
0619 02:49:09.491050 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:09.491052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:09.491061 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:49:09.491081 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:09.491082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:09.491091 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:09.491169 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:09.491891 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:09.492654 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:09.493683 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:09.493982 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:09.494047 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:09.495438 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:09.495439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:09.495439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:09.495446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:09.496973 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:09.497290 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:09.497291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:09.497291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:09.497346 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:09.497518 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:09.497519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:09.497519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:09.497626 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:09.497705 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:09.497804 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 02:49:09.497878 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:49:09.517721 8025 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:09.518302 8025 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:09.523777 8025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d84c6dc0a5634bc398f5d62f410b5dad: Generated new TSK 0
I20260619 02:49:09.523947 8025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:09.531330 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 7938
2026-06-19T02:49:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1229 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:09 dist-test-slave-zqpb krb5kdc[8037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:09 dist-test-slave-zqpb krb5kdc[8037](info): set up 2 sockets
Jun 19 02:49:09 dist-test-slave-zqpb krb5kdc[8037](info): commencing operation
krb5kdc: starting...
W20260619 02:49:10.482761 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:10 dist-test-slave-zqpb krb5kdc[8037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:10.592983 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38457
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:10.670940 8050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:10.671134 8050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:10.671156 8050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:10.672304 8050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:10.672343 8050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:10.672356 8050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:10.672367 8050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:10.672379 8050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:10.673825 8050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:10.674052 8050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:10.674253 8050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:10.678772 8055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:10.678763 8056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:10.678763 8058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:10.678861 8050 server_base.cc:1034] running on GCE node
I20260619 02:49:10.679227 8050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:10.679491 8050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:10.680652 8050 hybrid_clock.cc:648] HybridClock initialized: now 1781837350680637 us; error 32 us; skew 500 ppm
Jun 19 02:49:10 dist-test-slave-zqpb krb5kdc[8037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837350, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:10.683050 8050 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:10.683466 8050 webserver.cc:460] Webserver started at http://127.0.0.1:43453/ using document root <none> and password file <none>
I20260619 02:49:10.683669 8050 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:10.683714 8050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:10.683838 8050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:10.684875 8050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "e21910d7654d4441ba781df3422aaa6a"
format_stamp: "Formatted at 2026-06-19 02:49:10 on dist-test-slave-zqpb"
server_key: "fdbe4f94cf4e9fdc4c832523c31553bf"
server_key_iv: "162800eb0dcd215edd0f16e3f6927dba"
server_key_version: "encryptionkey@0"
I20260619 02:49:10.685200 8050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "e21910d7654d4441ba781df3422aaa6a"
format_stamp: "Formatted at 2026-06-19 02:49:10 on dist-test-slave-zqpb"
server_key: "fdbe4f94cf4e9fdc4c832523c31553bf"
server_key_iv: "162800eb0dcd215edd0f16e3f6927dba"
server_key_version: "encryptionkey@0"
I20260619 02:49:10.686380 8050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:10.687157 8065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:10.687290 8050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:49:10.687371 8050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "e21910d7654d4441ba781df3422aaa6a"
format_stamp: "Formatted at 2026-06-19 02:49:10 on dist-test-slave-zqpb"
server_key: "fdbe4f94cf4e9fdc4c832523c31553bf"
server_key_iv: "162800eb0dcd215edd0f16e3f6927dba"
server_key_version: "encryptionkey@0"
I20260619 02:49:10.687434 8050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:10.701880 8050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:10.707307 8050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:10.707441 8050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:10.711068 8050 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38457
I20260619 02:49:10.711122 8117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38457 every 8 connection(s)
I20260619 02:49:10.711407 8050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:10.712177 8118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:10.714372 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a: Bootstrap starting.
I20260619 02:49:10.714936 8118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:10.715183 8118 log.cc:824] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:10.715888 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a: No bootstrap required, opened a new log
I20260619 02:49:10.717392 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8050
I20260619 02:49:10.717510 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:10.717685 19909 external_mini_cluster.cc:1428] Setting key d79465bee564b5f666a90f09e93f7995
I20260619 02:49:10.717655 8118 raft_consensus.cc:348] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21910d7654d4441ba781df3422aaa6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38457 } }
I20260619 02:49:10.717829 8118 raft_consensus.cc:374] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:10.717873 8118 raft_consensus.cc:729] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e21910d7654d4441ba781df3422aaa6a, State: Initialized, Role: FOLLOWER
I20260619 02:49:10.718003 8118 consensus_queue.cc:260] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21910d7654d4441ba781df3422aaa6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38457 } }
I20260619 02:49:10.718302 8118 raft_consensus.cc:388] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:10.718384 8118 raft_consensus.cc:482] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:10.718443 8118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:10.719432 8118 raft_consensus.cc:504] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21910d7654d4441ba781df3422aaa6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38457 } }
I20260619 02:49:10.719530 8118 leader_election.cc:302] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e21910d7654d4441ba781df3422aaa6a; no voters:
I20260619 02:49:10.719683 8118 leader_election.cc:288] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:10.719726 8123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:10.719918 8123 raft_consensus.cc:686] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [term 1 LEADER]: Becoming Leader. State: Replica: e21910d7654d4441ba781df3422aaa6a, State: Running, Role: LEADER
I20260619 02:49:10.719947 8118 sys_catalog.cc:564] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:10.720064 8123 consensus_queue.cc:237] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21910d7654d4441ba781df3422aaa6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38457 } }
I20260619 02:49:10.720443 8124 sys_catalog.cc:455] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e21910d7654d4441ba781df3422aaa6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21910d7654d4441ba781df3422aaa6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38457 } } }
I20260619 02:49:10.720517 8124 sys_catalog.cc:458] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:10.720703 8125 sys_catalog.cc:455] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e21910d7654d4441ba781df3422aaa6a. Latest consensus state: current_term: 1 leader_uuid: "e21910d7654d4441ba781df3422aaa6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21910d7654d4441ba781df3422aaa6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38457 } } }
I20260619 02:49:10.720999 8125 sys_catalog.cc:458] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:10.721128 8131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:10.721623 8131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:10.723098 8131 catalog_manager.cc:1347] Generated new cluster ID: 91d4c2d385a6434185a7808fd09183af
I20260619 02:49:10.723147 8131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:10.723659 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:10.718144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56273 (local address 127.0.0.1:38457)
0619 02:49:10.718302 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:10.718311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:10.718496 (+ 185us) server_negotiation.cc:407] Connection header received
0619 02:49:10.718623 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:10.718628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:10.718671 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:10.718775 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:10.719064 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:10.720189 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:10.721323 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:10.721907 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:10.722003 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:10.722788 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:10.722791 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:10.722792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:10.723009 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:10.723046 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:10.723062 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:10.723118 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:10.723374 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:49:10.723453 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260619 02:49:10.724479 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:10.726850 8131 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:10.727515 8131 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:10.730572 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:10.725897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32835 (local address 127.0.0.1:38457)
0619 02:49:10.725929 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:10.725931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:10.727011 (+ 1080us) server_negotiation.cc:407] Connection header received
0619 02:49:10.727093 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:10.727095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:10.727105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:10.727163 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:10.727284 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:10.728014 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:10.729378 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:10.729666 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:10.729721 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:10.730337 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:10.730338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:10.730339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:10.730343 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:10.730358 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:10.730362 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:10.730400 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:10.730458 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:49:10.730499 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":16}
I20260619 02:49:10.732510 8131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e21910d7654d4441ba781df3422aaa6a: Generated new TSK 0
I20260619 02:49:10.732666 8131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:10.737076 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8050
2026-06-19T02:49:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1204 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:10 dist-test-slave-zqpb krb5kdc[8149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:10 dist-test-slave-zqpb krb5kdc[8149](info): set up 2 sockets
Jun 19 02:49:10 dist-test-slave-zqpb krb5kdc[8149](info): commencing operation
krb5kdc: starting...
W20260619 02:49:11.686673 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:11 dist-test-slave-zqpb krb5kdc[8149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:11.797416 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:11.877365 8162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:11.877594 8162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:11.877617 8162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:11.878787 8162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:11.878826 8162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:11.878839 8162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:11.878852 8162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:11.878895 8162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:11.880393 8162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:11.880620 8162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:11.880815 8162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:11.885427 8167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:11.885521 8170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:11.885617 8168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:11.885535 8162 server_base.cc:1034] running on GCE node
I20260619 02:49:11.885874 8162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:11.886153 8162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:11.887301 8162 hybrid_clock.cc:648] HybridClock initialized: now 1781837351887284 us; error 32 us; skew 500 ppm
Jun 19 02:49:11 dist-test-slave-zqpb krb5kdc[8149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837351, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:11.889638 8162 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:11.890059 8162 webserver.cc:460] Webserver started at http://127.0.0.1:44617/ using document root <none> and password file <none>
I20260619 02:49:11.890250 8162 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:11.890295 8162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:11.890413 8162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:11.891497 8162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "cec248e7ff3047b5bf8dab2e414e94a1"
format_stamp: "Formatted at 2026-06-19 02:49:11 on dist-test-slave-zqpb"
server_key: "5002931ae3c8e17b676fca2b6e5c40bf"
server_key_iv: "558d143307117a2ffac5785770549474"
server_key_version: "encryptionkey@0"
I20260619 02:49:11.891824 8162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "cec248e7ff3047b5bf8dab2e414e94a1"
format_stamp: "Formatted at 2026-06-19 02:49:11 on dist-test-slave-zqpb"
server_key: "5002931ae3c8e17b676fca2b6e5c40bf"
server_key_iv: "558d143307117a2ffac5785770549474"
server_key_version: "encryptionkey@0"
I20260619 02:49:11.893131 8162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:11.893814 8177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:11.893996 8162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:11.894079 8162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "cec248e7ff3047b5bf8dab2e414e94a1"
format_stamp: "Formatted at 2026-06-19 02:49:11 on dist-test-slave-zqpb"
server_key: "5002931ae3c8e17b676fca2b6e5c40bf"
server_key_iv: "558d143307117a2ffac5785770549474"
server_key_version: "encryptionkey@0"
I20260619 02:49:11.894138 8162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:11.897356 8162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:11.899204 8162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:11.899324 8162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:11.903249 8162 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38293
I20260619 02:49:11.903291 8229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38293 every 8 connection(s)
I20260619 02:49:11.903630 8162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:11.904506 8230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:11.906818 8230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1: Bootstrap starting.
I20260619 02:49:11.907388 8230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:11.907629 8230 log.cc:824] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:11.908277 8230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1: No bootstrap required, opened a new log
I20260619 02:49:11.909814 8230 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec248e7ff3047b5bf8dab2e414e94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38293 } }
I20260619 02:49:11.909943 8230 raft_consensus.cc:374] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:11.910032 8230 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec248e7ff3047b5bf8dab2e414e94a1, State: Initialized, Role: FOLLOWER
I20260619 02:49:11.910173 8230 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec248e7ff3047b5bf8dab2e414e94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38293 } }
I20260619 02:49:11.910233 8230 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:11.910264 8230 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:11.910307 8230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:11.910790 8230 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec248e7ff3047b5bf8dab2e414e94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38293 } }
I20260619 02:49:11.910912 8230 leader_election.cc:302] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec248e7ff3047b5bf8dab2e414e94a1; no voters:
I20260619 02:49:11.911073 8230 leader_election.cc:288] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:11.911134 8233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:11.911281 8233 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [term 1 LEADER]: Becoming Leader. State: Replica: cec248e7ff3047b5bf8dab2e414e94a1, State: Running, Role: LEADER
I20260619 02:49:11.911310 8230 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:11.911430 8233 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec248e7ff3047b5bf8dab2e414e94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38293 } }
I20260619 02:49:11.911801 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8162
I20260619 02:49:11.911921 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:11.911978 8235 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec248e7ff3047b5bf8dab2e414e94a1. Latest consensus state: current_term: 1 leader_uuid: "cec248e7ff3047b5bf8dab2e414e94a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec248e7ff3047b5bf8dab2e414e94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38293 } } }
I20260619 02:49:11.912048 8235 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:11.912102 19909 external_mini_cluster.cc:1428] Setting key 7a28b930c9e2cb514d45e00144766a95
I20260619 02:49:11.912358 8234 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cec248e7ff3047b5bf8dab2e414e94a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec248e7ff3047b5bf8dab2e414e94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38293 } } }
I20260619 02:49:11.912408 8242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:11.912425 8234 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:11.912974 8242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:11.914383 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:11.912738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59317 (local address 127.0.0.1:38293)
0619 02:49:11.912907 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:11.912917 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:11.912950 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:49:11.913103 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:11.913111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:11.913144 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:11.913549 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:11.913990 (+ 441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:11.913993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:11.913993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:11.913996 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:11.914028 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:11.914043 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:11.914114 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:11.914197 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:49:11.914239 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260619 02:49:11.914778 8242 catalog_manager.cc:1347] Generated new cluster ID: 5a135e8a21b8473ead6036c51677dbcc
I20260619 02:49:11.914824 8242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:11.915228 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:11.917301 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:11.916412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45047 (local address 127.0.0.1:38293)
0619 02:49:11.916443 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:11.916445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:11.916547 (+ 102us) server_negotiation.cc:407] Connection header received
0619 02:49:11.916620 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:11.916621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:11.916625 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:11.916669 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:11.917127 (+ 458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:11.917128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:11.917129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:11.917133 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:11.917147 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:11.917151 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:11.917195 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:11.917233 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:49:11.917245 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:49:11.939949 8242 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:11.940196 8242 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:11.947570 8242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cec248e7ff3047b5bf8dab2e414e94a1: Generated new TSK 0
I20260619 02:49:11.947778 8242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:11.952615 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8162
2026-06-19T02:49:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1215 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:12 dist-test-slave-zqpb krb5kdc[8261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:12 dist-test-slave-zqpb krb5kdc[8261](info): set up 2 sockets
Jun 19 02:49:12 dist-test-slave-zqpb krb5kdc[8261](info): commencing operation
krb5kdc: starting...
W20260619 02:49:12.926678 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:13 dist-test-slave-zqpb krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:13.035266 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:13.113862 8274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:13.114074 8274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:13.114104 8274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:13.115293 8274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:13.115332 8274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:13.115351 8274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:13.115366 8274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:13.115398 8274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:13.116834 8274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:13.117077 8274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:13.117268 8274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:13.121850 8279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:13.121881 8274 server_base.cc:1034] running on GCE node
W20260619 02:49:13.121863 8280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:13.121881 8282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:13.122290 8274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:13.122563 8274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:13.123716 8274 hybrid_clock.cc:648] HybridClock initialized: now 1781837353123698 us; error 30 us; skew 500 ppm
Jun 19 02:49:13 dist-test-slave-zqpb krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837353, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:13.126134 8274 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:13.126552 8274 webserver.cc:460] Webserver started at http://127.0.0.1:44575/ using document root <none> and password file <none>
I20260619 02:49:13.126749 8274 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:13.126791 8274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:13.126935 8274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:13.127982 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "4897178980494a8d90cb30c16e0ae83f"
format_stamp: "Formatted at 2026-06-19 02:49:13 on dist-test-slave-zqpb"
server_key: "5eb93c44460e16b0793cbfddcdd84e46"
server_key_iv: "a1c6b8962414ce27ccfa72d8ec23a97d"
server_key_version: "encryptionkey@0"
I20260619 02:49:13.128276 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "4897178980494a8d90cb30c16e0ae83f"
format_stamp: "Formatted at 2026-06-19 02:49:13 on dist-test-slave-zqpb"
server_key: "5eb93c44460e16b0793cbfddcdd84e46"
server_key_iv: "a1c6b8962414ce27ccfa72d8ec23a97d"
server_key_version: "encryptionkey@0"
I20260619 02:49:13.129487 8274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:13.130533 8289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:13.130692 8274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:49:13.130781 8274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "4897178980494a8d90cb30c16e0ae83f"
format_stamp: "Formatted at 2026-06-19 02:49:13 on dist-test-slave-zqpb"
server_key: "5eb93c44460e16b0793cbfddcdd84e46"
server_key_iv: "a1c6b8962414ce27ccfa72d8ec23a97d"
server_key_version: "encryptionkey@0"
I20260619 02:49:13.130841 8274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:13.149098 8274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:13.151011 8274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:13.151129 8274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:13.154739 8274 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43821
I20260619 02:49:13.154778 8341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43821 every 8 connection(s)
I20260619 02:49:13.155117 8274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:13.155921 8342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:13.158201 8342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f: Bootstrap starting.
I20260619 02:49:13.158754 8342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:13.159047 8342 log.cc:824] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:13.159315 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8274
I20260619 02:49:13.159441 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:13.159621 19909 external_mini_cluster.cc:1428] Setting key 7493166e6c243c9a531695f7e7f2646c
I20260619 02:49:13.159789 8342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f: No bootstrap required, opened a new log
I20260619 02:49:13.161629 8342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4897178980494a8d90cb30c16e0ae83f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } }
I20260619 02:49:13.161761 8342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:13.161792 8342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4897178980494a8d90cb30c16e0ae83f, State: Initialized, Role: FOLLOWER
I20260619 02:49:13.161914 8342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4897178980494a8d90cb30c16e0ae83f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } }
I20260619 02:49:13.161993 8342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:13.162022 8342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:13.162068 8342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:13.162585 8342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4897178980494a8d90cb30c16e0ae83f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } }
I20260619 02:49:13.162674 8342 leader_election.cc:302] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4897178980494a8d90cb30c16e0ae83f; no voters:
I20260619 02:49:13.162820 8342 leader_election.cc:288] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:13.162858 8347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:13.163021 8347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [term 1 LEADER]: Becoming Leader. State: Replica: 4897178980494a8d90cb30c16e0ae83f, State: Running, Role: LEADER
I20260619 02:49:13.163156 8347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4897178980494a8d90cb30c16e0ae83f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } }
I20260619 02:49:13.163196 8342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:13.164407 8349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4897178980494a8d90cb30c16e0ae83f. Latest consensus state: current_term: 1 leader_uuid: "4897178980494a8d90cb30c16e0ae83f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4897178980494a8d90cb30c16e0ae83f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } } }
I20260619 02:49:13.164649 8349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [sys.catalog]: This master's current role is: LEADER
W20260619 02:49:13.164788 8362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:13.164888 8362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:13.164578 8348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4897178980494a8d90cb30c16e0ae83f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4897178980494a8d90cb30c16e0ae83f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } } }
I20260619 02:49:13.165500 8348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:13.165180 8364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:13.165759 8364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 02:49:13 dist-test-slave-zqpb krb5kdc[8261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:49:13.167236 8364 catalog_manager.cc:1347] Generated new cluster ID: 1f049bb184c941b7ae8619886a530a65
I20260619 02:49:13.167286 8364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:13.171190 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:13.160072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:38133 (local address 127.0.0.1:43821)
0619 02:49:13.160253 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:13.160261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:13.160292 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:49:13.160988 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:13.160995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:13.161044 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:13.161191 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:13.161782 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:13.163119 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:13.164138 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:13.164719 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:13.164823 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:13.167547 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:13.167552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:13.167552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:13.167564 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:13.169582 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:13.169962 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:13.169965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:13.169965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:13.170072 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:13.170298 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:13.170299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:13.170300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:13.170591 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:13.170751 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:13.170907 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:49:13.171000 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260619 02:49:13.172130 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:13.181622 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:13.173463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:56325 (local address 127.0.0.1:43821)
0619 02:49:13.173494 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:13.173496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:13.175093 (+ 1597us) server_negotiation.cc:407] Connection header received
0619 02:49:13.175126 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:13.175127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:13.175137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:13.175203 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:13.175742 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:13.176559 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:13.177477 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:13.177808 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:13.177873 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:13.179099 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:13.179100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:13.179101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:13.179108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:13.180542 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:13.180903 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:13.180904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:13.180904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:13.180960 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:13.181144 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:13.181145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:13.181145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:13.181267 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:13.181366 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:13.181476 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:49:13.181535 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 02:49:13.183871 8364 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:13.184355 8364 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:13.187757 8364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4897178980494a8d90cb30c16e0ae83f: Generated new TSK 0
I20260619 02:49:13.187956 8364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:13.195571 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8274
W20260619 02:49:13.199407 8259 connection.cc:535] client connection to 127.0.0.1:43821 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43821): Connection reset by peer (error 104)
2026-06-19T02:49:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:13 dist-test-slave-zqpb krb5kdc[8373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:13 dist-test-slave-zqpb krb5kdc[8373](info): set up 2 sockets
Jun 19 02:49:13 dist-test-slave-zqpb krb5kdc[8373](info): commencing operation
krb5kdc: starting...
W20260619 02:49:14.147330 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:14 dist-test-slave-zqpb krb5kdc[8373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:14.261516 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44003
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:14.338529 8386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:14.338732 8386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:14.338752 8386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:14.339856 8386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:14.339890 8386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:14.339900 8386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:14.339912 8386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:14.339922 8386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:14.341282 8386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:14.341463 8386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:14.341634 8386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:14.346156 8392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:14.346128 8391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:14.346160 8386 server_base.cc:1034] running on GCE node
W20260619 02:49:14.346094 8394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:14.346506 8386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:14.346760 8386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:14.347892 8386 hybrid_clock.cc:648] HybridClock initialized: now 1781837354347858 us; error 47 us; skew 500 ppm
Jun 19 02:49:14 dist-test-slave-zqpb krb5kdc[8373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837354, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:14.350068 8386 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:14.350449 8386 webserver.cc:460] Webserver started at http://127.0.0.1:43435/ using document root <none> and password file <none>
I20260619 02:49:14.350621 8386 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:14.350662 8386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:14.350750 8386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:14.351727 8386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "8090c10af01b4d98958fb43c93659d9d"
format_stamp: "Formatted at 2026-06-19 02:49:14 on dist-test-slave-zqpb"
server_key: "ccf27598bd91b1e6751e1ed73f785132"
server_key_iv: "b33fa9599711652ee31fe91786f26270"
server_key_version: "encryptionkey@0"
I20260619 02:49:14.352002 8386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "8090c10af01b4d98958fb43c93659d9d"
format_stamp: "Formatted at 2026-06-19 02:49:14 on dist-test-slave-zqpb"
server_key: "ccf27598bd91b1e6751e1ed73f785132"
server_key_iv: "b33fa9599711652ee31fe91786f26270"
server_key_version: "encryptionkey@0"
I20260619 02:49:14.353168 8386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:14.353750 8401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:14.353890 8386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:14.354008 8386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "8090c10af01b4d98958fb43c93659d9d"
format_stamp: "Formatted at 2026-06-19 02:49:14 on dist-test-slave-zqpb"
server_key: "ccf27598bd91b1e6751e1ed73f785132"
server_key_iv: "b33fa9599711652ee31fe91786f26270"
server_key_version: "encryptionkey@0"
I20260619 02:49:14.354067 8386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:14.364296 8386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:14.365922 8386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:14.366034 8386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:14.369684 8386 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44003
I20260619 02:49:14.369724 8453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44003 every 8 connection(s)
I20260619 02:49:14.370045 8386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:14.370858 8454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:14.373262 8454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d: Bootstrap starting.
I20260619 02:49:14.373909 8454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:14.374162 8454 log.cc:824] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:14.374856 8454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d: No bootstrap required, opened a new log
I20260619 02:49:14.375588 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8386
I20260619 02:49:14.375698 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:14.375864 19909 external_mini_cluster.cc:1428] Setting key e6d85fb297bb9bcc5f3434fd15527b18
I20260619 02:49:14.376837 8454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8090c10af01b4d98958fb43c93659d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44003 } }
I20260619 02:49:14.376992 8454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:14.377032 8454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8090c10af01b4d98958fb43c93659d9d, State: Initialized, Role: FOLLOWER
I20260619 02:49:14.377158 8454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8090c10af01b4d98958fb43c93659d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44003 } }
I20260619 02:49:14.377238 8454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:14.377261 8454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:14.377307 8454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:14.377841 8454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8090c10af01b4d98958fb43c93659d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44003 } }
I20260619 02:49:14.377936 8454 leader_election.cc:302] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8090c10af01b4d98958fb43c93659d9d; no voters:
I20260619 02:49:14.378101 8454 leader_election.cc:288] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:14.378135 8459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:14.378230 8459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [term 1 LEADER]: Becoming Leader. State: Replica: 8090c10af01b4d98958fb43c93659d9d, State: Running, Role: LEADER
I20260619 02:49:14.378353 8454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:14.378368 8459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8090c10af01b4d98958fb43c93659d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44003 } }
I20260619 02:49:14.378679 8459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8090c10af01b4d98958fb43c93659d9d. Latest consensus state: current_term: 1 leader_uuid: "8090c10af01b4d98958fb43c93659d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8090c10af01b4d98958fb43c93659d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44003 } } }
I20260619 02:49:14.378747 8459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:14.378921 8460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8090c10af01b4d98958fb43c93659d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8090c10af01b4d98958fb43c93659d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44003 } } }
I20260619 02:49:14.379034 8460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d [sys.catalog]: This master's current role is: LEADER
W20260619 02:49:14.380190 8475 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:14.380249 8475 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:14.380290 8463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:14.380430 8463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:14.381793 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:14.376366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:57099 (local address 127.0.0.1:44003)
0619 02:49:14.376533 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:14.376542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:14.376566 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:49:14.376795 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:14.376800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:14.376845 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:14.376948 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:14.377211 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:14.378707 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:14.379757 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:14.380281 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:14.380410 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:14.381116 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:14.381120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:14.381121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:14.381323 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:14.381361 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:14.381375 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:14.381451 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:14.381559 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:49:14.381587 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260619 02:49:14.382210 8463 catalog_manager.cc:1347] Generated new cluster ID: d4b9df5345c8418e90a3fed17a645dd0
I20260619 02:49:14.382269 8463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:14.382747 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:14.390094 8463 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:14.390741 8463 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:14.393520 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:14.387184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:44515 (local address 127.0.0.1:44003)
0619 02:49:14.387219 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:14.387220 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:14.390224 (+ 3004us) server_negotiation.cc:407] Connection header received
0619 02:49:14.390286 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:14.390287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:14.390298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:14.390356 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:14.390478 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:14.391242 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:14.392270 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:14.392566 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:14.392616 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:14.393292 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:14.393294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:14.393294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:14.393298 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:14.393312 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:14.393316 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:14.393367 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:14.393436 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:49:14.393448 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":17}
I20260619 02:49:14.394863 8463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8090c10af01b4d98958fb43c93659d9d: Generated new TSK 0
I20260619 02:49:14.395044 8463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:14.398710 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8386
W20260619 02:49:14.402597 8371 connection.cc:535] client connection to 127.0.0.1:44003 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44003): Connection reset by peer (error 104)
2026-06-19T02:49:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1203 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:14 dist-test-slave-zqpb krb5kdc[8485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:14 dist-test-slave-zqpb krb5kdc[8485](info): set up 2 sockets
Jun 19 02:49:14 dist-test-slave-zqpb krb5kdc[8485](info): commencing operation
krb5kdc: starting...
W20260619 02:49:15.354619 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:15 dist-test-slave-zqpb krb5kdc[8485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:15.463083 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:15.539537 8498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:15.539739 8498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:15.539769 8498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:15.540884 8498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:15.540923 8498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:15.540941 8498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:15.540957 8498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:15.540970 8498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:15.542380 8498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:15.542611 8498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:15.542802 8498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:15.547319 8506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:15.547425 8498 server_base.cc:1034] running on GCE node
W20260619 02:49:15.547348 8503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:15.547348 8504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:15.547739 8498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:15.547999 8498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:15.549135 8498 hybrid_clock.cc:648] HybridClock initialized: now 1781837355549107 us; error 44 us; skew 500 ppm
Jun 19 02:49:15 dist-test-slave-zqpb krb5kdc[8485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837355, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:15.555872 8498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:15.556286 8498 webserver.cc:460] Webserver started at http://127.0.0.1:44919/ using document root <none> and password file <none>
I20260619 02:49:15.556453 8498 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:15.556489 8498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:15.556597 8498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:15.557559 8498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "34c78764de614579935cfad5b6f14c4c"
format_stamp: "Formatted at 2026-06-19 02:49:15 on dist-test-slave-zqpb"
server_key: "85909b482f05b8968eb5d5a97e8da366"
server_key_iv: "9ad25cf96927f436495400fab0e9f79b"
server_key_version: "encryptionkey@0"
I20260619 02:49:15.557844 8498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "34c78764de614579935cfad5b6f14c4c"
format_stamp: "Formatted at 2026-06-19 02:49:15 on dist-test-slave-zqpb"
server_key: "85909b482f05b8968eb5d5a97e8da366"
server_key_iv: "9ad25cf96927f436495400fab0e9f79b"
server_key_version: "encryptionkey@0"
I20260619 02:49:15.559067 8498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:15.559757 8513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:15.559931 8498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:15.560015 8498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "34c78764de614579935cfad5b6f14c4c"
format_stamp: "Formatted at 2026-06-19 02:49:15 on dist-test-slave-zqpb"
server_key: "85909b482f05b8968eb5d5a97e8da366"
server_key_iv: "9ad25cf96927f436495400fab0e9f79b"
server_key_version: "encryptionkey@0"
I20260619 02:49:15.560068 8498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:15.563174 8498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:15.564774 8498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:15.564869 8498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:15.568439 8498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43813
I20260619 02:49:15.568468 8565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43813 every 8 connection(s)
I20260619 02:49:15.568753 8498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:15.569566 8566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:15.571787 8566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c: Bootstrap starting.
I20260619 02:49:15.572299 8566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:15.572546 8566 log.cc:824] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:15.573140 8566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c: No bootstrap required, opened a new log
I20260619 02:49:15.574692 8566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c78764de614579935cfad5b6f14c4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260619 02:49:15.574822 8566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:15.574853 8566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c78764de614579935cfad5b6f14c4c, State: Initialized, Role: FOLLOWER
I20260619 02:49:15.575008 8566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c78764de614579935cfad5b6f14c4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260619 02:49:15.575081 8566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:15.575114 8566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:15.575165 8566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:15.575675 8566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c78764de614579935cfad5b6f14c4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260619 02:49:15.575766 8566 leader_election.cc:302] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34c78764de614579935cfad5b6f14c4c; no voters:
I20260619 02:49:15.575925 8566 leader_election.cc:288] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:15.575963 8569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:15.576093 8569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [term 1 LEADER]: Becoming Leader. State: Replica: 34c78764de614579935cfad5b6f14c4c, State: Running, Role: LEADER
I20260619 02:49:15.576156 8566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:15.576177 8569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c78764de614579935cfad5b6f14c4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260619 02:49:15.576540 8571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34c78764de614579935cfad5b6f14c4c. Latest consensus state: current_term: 1 leader_uuid: "34c78764de614579935cfad5b6f14c4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c78764de614579935cfad5b6f14c4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } } }
I20260619 02:49:15.576670 8571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:15.576910 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8498
I20260619 02:49:15.577010 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:15.577126 19909 external_mini_cluster.cc:1428] Setting key afbab162052f92bca49fff8354a7894c
I20260619 02:49:15.576660 8570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34c78764de614579935cfad5b6f14c4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c78764de614579935cfad5b6f14c4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } } }
I20260619 02:49:15.577355 8570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c [sys.catalog]: This master's current role is: LEADER
W20260619 02:49:15.578188 8585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:15.578244 8585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:15.578336 8584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:15.578504 8584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:15.579064 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:15.577571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:53135 (local address 127.0.0.1:43813)
0619 02:49:15.577786 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:15.577794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:15.577821 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:49:15.577904 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:15.577910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:15.577931 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:15.578044 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:15.578633 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:15.578636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:15.578636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:15.578640 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:15.578675 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:15.578685 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:15.578738 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:15.578809 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:49:15.578850 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":58,"threads_started":1}
I20260619 02:49:15.579912 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:15.581295 8584 catalog_manager.cc:1347] Generated new cluster ID: ab563f7b1b21447cb7215a17988325d3
I20260619 02:49:15.581367 8584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:15.583767 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:15.581355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:38079 (local address 127.0.0.1:43813)
0619 02:49:15.581393 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:15.581395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:15.582996 (+ 1601us) server_negotiation.cc:407] Connection header received
0619 02:49:15.583052 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:15.583053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:15.583058 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:15.583103 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:15.583565 (+ 462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:15.583566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:15.583566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:15.583570 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:15.583584 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:15.583595 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:15.583642 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:15.583696 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 02:49:15.583710 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:49:15.620613 8584 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:15.620787 8584 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:15.623448 8584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34c78764de614579935cfad5b6f14c4c: Generated new TSK 0
I20260619 02:49:15.623624 8584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:15.629334 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8498
2026-06-19T02:49:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1230 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:15 dist-test-slave-zqpb krb5kdc[8597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:15 dist-test-slave-zqpb krb5kdc[8597](info): set up 2 sockets
Jun 19 02:49:15 dist-test-slave-zqpb krb5kdc[8597](info): commencing operation
krb5kdc: starting...
W20260619 02:49:16.578780 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:16 dist-test-slave-zqpb krb5kdc[8597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:16.689013 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:16.769595 8610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:16.769793 8610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:16.769812 8610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:16.770982 8610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:16.771020 8610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:16.771034 8610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:16.771044 8610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:16.771054 8610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:16.772467 8610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:16.772652 8610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:16.772871 8610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:16.777436 8615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:16.777457 8610 server_base.cc:1034] running on GCE node
W20260619 02:49:16.777690 8618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:16.777449 8616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:16.778025 8610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:16.778318 8610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:16.779474 8610 hybrid_clock.cc:648] HybridClock initialized: now 1781837356779457 us; error 27 us; skew 500 ppm
Jun 19 02:49:16 dist-test-slave-zqpb krb5kdc[8597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837356, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:16.781860 8610 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:16.782320 8610 webserver.cc:460] Webserver started at http://127.0.0.1:33139/ using document root <none> and password file <none>
I20260619 02:49:16.782541 8610 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:16.782590 8610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:16.782783 8610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:16.783847 8610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "a1e3694e5d274fc79b9a83c747fbd268"
format_stamp: "Formatted at 2026-06-19 02:49:16 on dist-test-slave-zqpb"
server_key: "11c596b443702cbea48d9f29e210758e"
server_key_iv: "d438f6595404927a5d8a8b0d94ac2148"
server_key_version: "encryptionkey@0"
I20260619 02:49:16.784152 8610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "a1e3694e5d274fc79b9a83c747fbd268"
format_stamp: "Formatted at 2026-06-19 02:49:16 on dist-test-slave-zqpb"
server_key: "11c596b443702cbea48d9f29e210758e"
server_key_iv: "d438f6595404927a5d8a8b0d94ac2148"
server_key_version: "encryptionkey@0"
I20260619 02:49:16.785363 8610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:16.786022 8625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:16.786201 8610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:16.786288 8610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "a1e3694e5d274fc79b9a83c747fbd268"
format_stamp: "Formatted at 2026-06-19 02:49:16 on dist-test-slave-zqpb"
server_key: "11c596b443702cbea48d9f29e210758e"
server_key_iv: "d438f6595404927a5d8a8b0d94ac2148"
server_key_version: "encryptionkey@0"
I20260619 02:49:16.786358 8610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:16.794981 8610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:16.796550 8610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:16.796669 8610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:16.800576 8610 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44359
I20260619 02:49:16.800604 8677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44359 every 8 connection(s)
I20260619 02:49:16.800956 8610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:16.801739 8678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:16.803232 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8610
I20260619 02:49:16.803345 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:16.803525 19909 external_mini_cluster.cc:1428] Setting key 3befbc9e695a06948ea7b503c83a5fa4
I20260619 02:49:16.804086 8678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268: Bootstrap starting.
I20260619 02:49:16.804802 8678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:16.805132 8678 log.cc:824] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:16.805868 8678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268: No bootstrap required, opened a new log
I20260619 02:49:16.808020 8678 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e3694e5d274fc79b9a83c747fbd268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260619 02:49:16.808182 8678 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:16.808213 8678 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e3694e5d274fc79b9a83c747fbd268, State: Initialized, Role: FOLLOWER
I20260619 02:49:16.808332 8678 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e3694e5d274fc79b9a83c747fbd268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260619 02:49:16.808403 8678 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:16.808439 8678 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:16.808485 8678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:16.809109 8678 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e3694e5d274fc79b9a83c747fbd268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260619 02:49:16.809197 8678 leader_election.cc:302] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1e3694e5d274fc79b9a83c747fbd268; no voters:
I20260619 02:49:16.809348 8678 leader_election.cc:288] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:16.809414 8683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:16.809608 8678 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:16.809610 8683 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [term 1 LEADER]: Becoming Leader. State: Replica: a1e3694e5d274fc79b9a83c747fbd268, State: Running, Role: LEADER
I20260619 02:49:16.809768 8683 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e3694e5d274fc79b9a83c747fbd268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260619 02:49:16.810111 8685 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1e3694e5d274fc79b9a83c747fbd268. Latest consensus state: current_term: 1 leader_uuid: "a1e3694e5d274fc79b9a83c747fbd268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e3694e5d274fc79b9a83c747fbd268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } } }
I20260619 02:49:16.810194 8685 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:16.810300 8684 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1e3694e5d274fc79b9a83c747fbd268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e3694e5d274fc79b9a83c747fbd268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } } }
I20260619 02:49:16.810434 8684 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268 [sys.catalog]: This master's current role is: LEADER
Jun 19 02:49:16 dist-test-slave-zqpb krb5kdc[8597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260619 02:49:16.811484 8698 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:16.811535 8698 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:16.811640 8692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:16.811823 8692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:16.813243 8692 catalog_manager.cc:1347] Generated new cluster ID: 124efc138d6b41a1a43d60f254771f8b
I20260619 02:49:16.813292 8692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:16.815284 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:16.804008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:43843 (local address 127.0.0.1:44359)
0619 02:49:16.804190 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:16.804198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:16.804229 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:49:16.804757 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:16.804766 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:16.804828 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:16.804967 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:16.805690 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:16.807094 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:16.807810 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:16.808405 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:16.808506 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:16.811361 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:16.811364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:16.811365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:16.811376 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:16.813646 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:16.814073 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:16.814076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:16.814077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:16.814184 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:16.814386 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:16.814387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:16.814387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:16.814689 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:16.814857 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:16.814978 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 02:49:16.815079 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260619 02:49:16.816160 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:16.825572 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:16.817633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:47345 (local address 127.0.0.1:44359)
0619 02:49:16.817664 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:16.817665 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:16.819014 (+ 1349us) server_negotiation.cc:407] Connection header received
0619 02:49:16.819058 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:16.819059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:16.819069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:16.819130 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:16.819671 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:16.820425 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:16.821356 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:16.821676 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:16.821736 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:16.822980 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:16.822981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:16.822982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:16.822989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:16.824403 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:16.824780 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:16.824781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:16.824781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:16.824834 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:16.825005 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:16.825006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:16.825006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:16.825125 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:16.825220 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:16.825319 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 02:49:16.825379 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 02:49:16.829789 8692 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:16.830266 8692 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:16.838889 8692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1e3694e5d274fc79b9a83c747fbd268: Generated new TSK 0
I20260619 02:49:16.839079 8692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:16.842017 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8610
W20260619 02:49:16.845826 8595 connection.cc:535] client connection to 127.0.0.1:44359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44359): Connection reset by peer (error 104)
2026-06-19T02:49:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1212 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:16 dist-test-slave-zqpb krb5kdc[8709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:16 dist-test-slave-zqpb krb5kdc[8709](info): set up 2 sockets
Jun 19 02:49:16 dist-test-slave-zqpb krb5kdc[8709](info): commencing operation
krb5kdc: starting...
W20260619 02:49:17.790699 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:17 dist-test-slave-zqpb krb5kdc[8709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:17.897169 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:17.973987 8722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:17.974175 8722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:17.974208 8722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:17.975340 8722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:17.975379 8722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:17.975397 8722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:17.975414 8722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:17.975427 8722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:17.976823 8722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:17.977065 8722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:17.977257 8722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:17.981716 8730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:17.981760 8727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:17.981838 8722 server_base.cc:1034] running on GCE node
W20260619 02:49:17.981738 8728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:17.982079 8722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:17.982324 8722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:17.983464 8722 hybrid_clock.cc:648] HybridClock initialized: now 1781837357983446 us; error 30 us; skew 500 ppm
Jun 19 02:49:17 dist-test-slave-zqpb krb5kdc[8709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837357, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:17.985740 8722 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:17.986155 8722 webserver.cc:460] Webserver started at http://127.0.0.1:39441/ using document root <none> and password file <none>
I20260619 02:49:17.986346 8722 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:17.986392 8722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:17.986510 8722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:17.987563 8722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "d5bb6d7b1e64468c898c0b027b9e718b"
format_stamp: "Formatted at 2026-06-19 02:49:17 on dist-test-slave-zqpb"
server_key: "877c5e1d60ed943922651c3392346b04"
server_key_iv: "e4d625aa723888c7debc5e0d5af274a1"
server_key_version: "encryptionkey@0"
I20260619 02:49:17.987905 8722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "d5bb6d7b1e64468c898c0b027b9e718b"
format_stamp: "Formatted at 2026-06-19 02:49:17 on dist-test-slave-zqpb"
server_key: "877c5e1d60ed943922651c3392346b04"
server_key_iv: "e4d625aa723888c7debc5e0d5af274a1"
server_key_version: "encryptionkey@0"
I20260619 02:49:17.989120 8722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:17.989748 8737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:17.989894 8722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:17.989975 8722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "d5bb6d7b1e64468c898c0b027b9e718b"
format_stamp: "Formatted at 2026-06-19 02:49:17 on dist-test-slave-zqpb"
server_key: "877c5e1d60ed943922651c3392346b04"
server_key_iv: "e4d625aa723888c7debc5e0d5af274a1"
server_key_version: "encryptionkey@0"
I20260619 02:49:17.990036 8722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:18.003639 8722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:18.005241 8722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:18.005368 8722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:18.008953 8722 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40029
I20260619 02:49:18.008985 8789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40029 every 8 connection(s)
I20260619 02:49:18.009284 8722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:18.010054 8790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:18.011022 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8722
I20260619 02:49:18.011142 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:18.011304 19909 external_mini_cluster.cc:1428] Setting key ad5674374ac7be13084f3619b81e412e
I20260619 02:49:18.012712 8790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b: Bootstrap starting.
I20260619 02:49:18.013415 8790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:18.013731 8790 log.cc:824] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:18.014392 8790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b: No bootstrap required, opened a new log
I20260619 02:49:18.016611 8790 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5bb6d7b1e64468c898c0b027b9e718b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } }
I20260619 02:49:18.016777 8790 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:18.016819 8790 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bb6d7b1e64468c898c0b027b9e718b, State: Initialized, Role: FOLLOWER
I20260619 02:49:18.016839 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:18.011738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:52841 (local address 127.0.0.1:40029)
0619 02:49:18.011927 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:18.011936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:18.011967 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:49:18.012457 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:18.012466 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:18.012529 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:18.012674 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:18.012815 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:18.014087 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:18.014858 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:18.015473 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:18.015579 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:18.016199 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:18.016204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:18.016204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:18.016409 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:18.016450 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:18.016465 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:18.016551 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:18.016616 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 02:49:18.016645 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260619 02:49:18.016963 8790 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5bb6d7b1e64468c898c0b027b9e718b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } }
I20260619 02:49:18.017035 8790 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:18.017063 8790 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:18.017110 8790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:18.017706 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:18.017763 8790 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5bb6d7b1e64468c898c0b027b9e718b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } }
I20260619 02:49:18.017866 8790 leader_election.cc:302] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5bb6d7b1e64468c898c0b027b9e718b; no voters:
I20260619 02:49:18.018018 8790 leader_election.cc:288] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:18.018059 8795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:18.018239 8790 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:18.018252 8795 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [term 1 LEADER]: Becoming Leader. State: Replica: d5bb6d7b1e64468c898c0b027b9e718b, State: Running, Role: LEADER
I20260619 02:49:18.018461 8795 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5bb6d7b1e64468c898c0b027b9e718b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } }
I20260619 02:49:18.018844 8797 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5bb6d7b1e64468c898c0b027b9e718b. Latest consensus state: current_term: 1 leader_uuid: "d5bb6d7b1e64468c898c0b027b9e718b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5bb6d7b1e64468c898c0b027b9e718b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } } }
I20260619 02:49:18.018939 8797 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:18.019225 8796 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5bb6d7b1e64468c898c0b027b9e718b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5bb6d7b1e64468c898c0b027b9e718b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } } }
I20260619 02:49:18.019282 8796 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:18.019476 8808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:18.019953 8808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:18.021355 8808 catalog_manager.cc:1347] Generated new cluster ID: c0d285d3bcec4ba5a1c002630911f2d1
I20260619 02:49:18.021399 8808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:18.022545 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:18.019225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:59047 (local address 127.0.0.1:40029)
0619 02:49:18.019260 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:18.019261 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:18.019391 (+ 130us) server_negotiation.cc:407] Connection header received
0619 02:49:18.019491 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:18.019492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:18.019504 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:18.019564 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:18.019785 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:18.020469 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:18.021332 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:18.021612 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:18.021650 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:18.022321 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:18.022322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:18.022323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:18.022327 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:18.022343 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:18.022347 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:18.022410 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:18.022469 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:49:18.022479 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:49:18.034672 8808 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:18.035235 8808 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:18.039983 8808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5bb6d7b1e64468c898c0b027b9e718b: Generated new TSK 0
I20260619 02:49:18.040162 8808 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:18.042088 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8722
W20260619 02:49:18.045674 8707 connection.cc:535] client connection to 127.0.0.1:40029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40029): Connection reset by peer (error 104)
2026-06-19T02:49:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1199 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:18 dist-test-slave-zqpb krb5kdc[8821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:18 dist-test-slave-zqpb krb5kdc[8821](info): set up 2 sockets
Jun 19 02:49:18 dist-test-slave-zqpb krb5kdc[8821](info): commencing operation
krb5kdc: starting...
W20260619 02:49:18.986721 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:19 dist-test-slave-zqpb krb5kdc[8821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:49:19.095674 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:19.172482 8834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:19.172670 8834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:19.172691 8834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:19.173741 8834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:19.173771 8834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:19.173784 8834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:19.173795 8834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:19.173805 8834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:19.175194 8834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34757
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:19.175365 8834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:19.175555 8834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:19.179970 8840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:19.179970 8839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:19.180020 8842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:19.180150 8834 server_base.cc:1034] running on GCE node
I20260619 02:49:19.180387 8834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:19.180648 8834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:19.181782 8834 hybrid_clock.cc:648] HybridClock initialized: now 1781837359181768 us; error 28 us; skew 500 ppm
Jun 19 02:49:19 dist-test-slave-zqpb krb5kdc[8821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837359, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:19.184019 8834 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:49:19.184438 8834 webserver.cc:460] Webserver started at http://127.0.0.1:40451/ using document root <none> and password file <none>
I20260619 02:49:19.184652 8834 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:19.184698 8834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:19.184818 8834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:19.185811 8834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "d28e780da8394933ae2c47cd0400a79c"
format_stamp: "Formatted at 2026-06-19 02:49:19 on dist-test-slave-zqpb"
server_key: "f8fb4ce7e51b27d7a25da8c19babdd61"
server_key_iv: "d86a86122276d75d5b8ce15f87223e79"
server_key_version: "encryptionkey@0"
I20260619 02:49:19.186118 8834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "d28e780da8394933ae2c47cd0400a79c"
format_stamp: "Formatted at 2026-06-19 02:49:19 on dist-test-slave-zqpb"
server_key: "f8fb4ce7e51b27d7a25da8c19babdd61"
server_key_iv: "d86a86122276d75d5b8ce15f87223e79"
server_key_version: "encryptionkey@0"
I20260619 02:49:19.187382 8834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:19.188136 8849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:19.188346 8834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:49:19.188423 8834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "d28e780da8394933ae2c47cd0400a79c"
format_stamp: "Formatted at 2026-06-19 02:49:19 on dist-test-slave-zqpb"
server_key: "f8fb4ce7e51b27d7a25da8c19babdd61"
server_key_iv: "d86a86122276d75d5b8ce15f87223e79"
server_key_version: "encryptionkey@0"
I20260619 02:49:19.188482 8834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:19.191668 8834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:19.193202 8834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:19.193317 8834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:19.196887 8834 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34757
I20260619 02:49:19.196919 8901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34757 every 8 connection(s)
I20260619 02:49:19.197185 8834 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:19.197978 8902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:19.199419 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8834
I20260619 02:49:19.199507 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:19.199649 19909 external_mini_cluster.cc:1428] Setting key d2d166cdcf310dfd887782ebb181f74b
I20260619 02:49:19.200343 8902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c: Bootstrap starting.
I20260619 02:49:19.200901 8902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:19.201222 8902 log.cc:824] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:19.201853 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:19.200086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:35579 (local address 127.0.0.1:34757)
0619 02:49:19.200233 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:19.200240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:19.200263 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:49:19.200672 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:19.200679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:19.200704 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:19.200813 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:19.201517 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:19.201520 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:19.201520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:19.201524 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:19.201554 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:19.201564 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:19.201642 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:19.201673 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 02:49:19.201712 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260619 02:49:19.201887 8902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c: No bootstrap required, opened a new log
I20260619 02:49:19.202762 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:19.203904 8902 raft_consensus.cc:348] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28e780da8394933ae2c47cd0400a79c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } }
I20260619 02:49:19.204105 8902 raft_consensus.cc:374] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:19.204159 8902 raft_consensus.cc:729] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d28e780da8394933ae2c47cd0400a79c, State: Initialized, Role: FOLLOWER
I20260619 02:49:19.204272 8902 consensus_queue.cc:260] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28e780da8394933ae2c47cd0400a79c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } }
I20260619 02:49:19.204367 8902 raft_consensus.cc:388] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:19.204401 8902 raft_consensus.cc:482] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:19.204443 8902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:19.204979 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:19.204036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.109:39875 (local address 127.0.0.1:34757)
0619 02:49:19.204066 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:19.204068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:19.204194 (+ 126us) server_negotiation.cc:407] Connection header received
0619 02:49:19.204274 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:19.204275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:19.204279 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:19.204342 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:19.204804 (+ 462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:19.204805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:19.204806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:19.204809 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:19.204824 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:19.204828 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:19.204880 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:19.204911 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 02:49:19.204923 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 02:49:19.205118 8902 raft_consensus.cc:504] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28e780da8394933ae2c47cd0400a79c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } }
I20260619 02:49:19.205214 8902 leader_election.cc:302] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d28e780da8394933ae2c47cd0400a79c; no voters:
I20260619 02:49:19.205367 8902 leader_election.cc:288] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:19.205399 8912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:19.205533 8912 raft_consensus.cc:686] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [term 1 LEADER]: Becoming Leader. State: Replica: d28e780da8394933ae2c47cd0400a79c, State: Running, Role: LEADER
I20260619 02:49:19.205595 8902 sys_catalog.cc:564] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:19.205638 8912 consensus_queue.cc:237] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28e780da8394933ae2c47cd0400a79c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } }
I20260619 02:49:19.205983 8914 sys_catalog.cc:455] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d28e780da8394933ae2c47cd0400a79c. Latest consensus state: current_term: 1 leader_uuid: "d28e780da8394933ae2c47cd0400a79c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28e780da8394933ae2c47cd0400a79c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } } }
I20260619 02:49:19.205973 8913 sys_catalog.cc:455] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d28e780da8394933ae2c47cd0400a79c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28e780da8394933ae2c47cd0400a79c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } } }
I20260619 02:49:19.206099 8914 sys_catalog.cc:458] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:19.206290 8920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:19.206104 8913 sys_catalog.cc:458] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:19.206795 8920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:19.208196 8920 catalog_manager.cc:1347] Generated new cluster ID: 117636e4cbb941109130678fa96c0a50
I20260619 02:49:19.208246 8920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:19.219236 8920 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:19.219388 8920 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:19.224012 8920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d28e780da8394933ae2c47cd0400a79c: Generated new TSK 0
I20260619 02:49:19.224161 8920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:19.229652 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8834
2026-06-19T02:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1187 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:19 dist-test-slave-zqpb krb5kdc[8933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:19 dist-test-slave-zqpb krb5kdc[8933](info): set up 2 sockets
Jun 19 02:49:19 dist-test-slave-zqpb krb5kdc[8933](info): commencing operation
krb5kdc: starting...
W20260619 02:49:20.182807 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:20 dist-test-slave-zqpb krb5kdc[8933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:20.293494 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:38679
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:38679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:20.372148 8946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:20.372350 8946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:20.372380 8946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:20.373530 8946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:20.373570 8946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:20.373589 8946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:20.373605 8946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:20.373621 8946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:20.375114 8946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:38679
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:38679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:20.375346 8946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:20.375550 8946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:20.380121 8951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:20.380136 8954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:20.380138 8952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:20.380152 8946 server_base.cc:1034] running on GCE node
I20260619 02:49:20.380525 8946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:20.380756 8946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:20.381908 8946 hybrid_clock.cc:648] HybridClock initialized: now 1781837360381891 us; error 27 us; skew 500 ppm
Jun 19 02:49:20 dist-test-slave-zqpb krb5kdc[8933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837360, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:20.384310 8946 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:20.384727 8946 webserver.cc:460] Webserver started at http://127.19.113.126:38807/ using document root <none> and password file <none>
I20260619 02:49:20.384936 8946 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:20.384982 8946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:20.385113 8946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:20.386116 8946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "15484ac85f074d7fa3ee6d2d4142f528"
format_stamp: "Formatted at 2026-06-19 02:49:20 on dist-test-slave-zqpb"
server_key: "74c328fb8b3f2fa2257d6c5cac830f4b"
server_key_iv: "bd20d997f3b8bd659693766ed50d31e2"
server_key_version: "encryptionkey@0"
I20260619 02:49:20.386428 8946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "15484ac85f074d7fa3ee6d2d4142f528"
format_stamp: "Formatted at 2026-06-19 02:49:20 on dist-test-slave-zqpb"
server_key: "74c328fb8b3f2fa2257d6c5cac830f4b"
server_key_iv: "bd20d997f3b8bd659693766ed50d31e2"
server_key_version: "encryptionkey@0"
I20260619 02:49:20.387775 8946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:49:20.388607 8961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:20.388803 8946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:49:20.388888 8946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "15484ac85f074d7fa3ee6d2d4142f528"
format_stamp: "Formatted at 2026-06-19 02:49:20 on dist-test-slave-zqpb"
server_key: "74c328fb8b3f2fa2257d6c5cac830f4b"
server_key_iv: "bd20d997f3b8bd659693766ed50d31e2"
server_key_version: "encryptionkey@0"
I20260619 02:49:20.388986 8946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:20.405802 8946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:20.407929 8946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:20.408062 8946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:20.411823 8946 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:38679
I20260619 02:49:20.411854 9013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:38679 every 8 connection(s)
I20260619 02:49:20.412212 8946 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:20.413025 9014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:20.415439 9014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528: Bootstrap starting.
I20260619 02:49:20.415966 9014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:20.416194 9014 log.cc:824] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:20.416842 9014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528: No bootstrap required, opened a new log
I20260619 02:49:20.418031 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 8946
I20260619 02:49:20.418164 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:20.418318 19909 external_mini_cluster.cc:1428] Setting key 5ee902d1a11505880f57467686a92561
I20260619 02:49:20.418624 9014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15484ac85f074d7fa3ee6d2d4142f528" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38679 } }
I20260619 02:49:20.418892 9014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:20.418954 9014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15484ac85f074d7fa3ee6d2d4142f528, State: Initialized, Role: FOLLOWER
I20260619 02:49:20.419080 9014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15484ac85f074d7fa3ee6d2d4142f528" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38679 } }
I20260619 02:49:20.419150 9014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:20.419174 9014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:20.419221 9014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:20.419914 9014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15484ac85f074d7fa3ee6d2d4142f528" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38679 } }
I20260619 02:49:20.420028 9014 leader_election.cc:302] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15484ac85f074d7fa3ee6d2d4142f528; no voters:
I20260619 02:49:20.420198 9014 leader_election.cc:288] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:20.420236 9019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:20.420406 9019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [term 1 LEADER]: Becoming Leader. State: Replica: 15484ac85f074d7fa3ee6d2d4142f528, State: Running, Role: LEADER
I20260619 02:49:20.420467 9014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:20.420567 9019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15484ac85f074d7fa3ee6d2d4142f528" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38679 } }
I20260619 02:49:20.420914 9019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15484ac85f074d7fa3ee6d2d4142f528. Latest consensus state: current_term: 1 leader_uuid: "15484ac85f074d7fa3ee6d2d4142f528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15484ac85f074d7fa3ee6d2d4142f528" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38679 } } }
I20260619 02:49:20.420984 9021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15484ac85f074d7fa3ee6d2d4142f528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15484ac85f074d7fa3ee6d2d4142f528" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 38679 } } }
I20260619 02:49:20.421033 9019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:20.421100 9021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:20.421270 9027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:20.421761 9027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:20.423280 9027 catalog_manager.cc:1347] Generated new cluster ID: dca81ea1d0a14563bc57ea0ce5428e94
I20260619 02:49:20.423331 9027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:49:20 dist-test-slave-zqpb krb5kdc[8933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:20.429548 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:20.418859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52369 (local address 127.19.113.126:38679)
0619 02:49:20.419133 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:20.419143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:20.419177 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:49:20.419312 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:20.419318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:20.419365 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:20.419474 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:20.420278 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:20.421707 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:20.422769 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:20.423371 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:20.423467 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:20.425884 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:20.425887 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:20.425888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:20.425900 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:20.427847 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:20.428287 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:20.428290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:20.428290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:20.428413 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:20.428636 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:20.428637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:20.428637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:20.428966 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:20.429136 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:20.429214 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 02:49:20.429317 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":91,"threads_started":1}
I20260619 02:49:20.430637 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:20.439188 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:20.432054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46409 (local address 127.19.113.126:38679)
0619 02:49:20.432091 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:20.432093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:20.432252 (+ 159us) server_negotiation.cc:407] Connection header received
0619 02:49:20.432278 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:20.432279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:20.432290 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:20.432351 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:20.432971 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:20.433762 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:20.434783 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:20.435135 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:20.435204 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:20.436539 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:20.436540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:20.436541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:20.436548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:20.438052 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:20.438431 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:20.438432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:20.438433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:20.438491 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:20.438675 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:20.438676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:20.438676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:20.438803 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:20.438954 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:20.439031 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:49:20.439095 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:49:20.446466 9027 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:20.447165 9027 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:20.455456 9027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15484ac85f074d7fa3ee6d2d4142f528: Generated new TSK 0
I20260619 02:49:20.455631 9027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:20.461621 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 8946
W20260619 02:49:20.465304 8931 connection.cc:535] client connection to 127.19.113.126:38679 recv error: Network error: failed to read from TLS socket (remote: 127.19.113.126:38679): Connection reset by peer (error 104)
2026-06-19T02:49:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1232 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:20 dist-test-slave-zqpb krb5kdc[9045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:20 dist-test-slave-zqpb krb5kdc[9045](info): set up 2 sockets
Jun 19 02:49:20 dist-test-slave-zqpb krb5kdc[9045](info): commencing operation
krb5kdc: starting...
W20260619 02:49:21.414696 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:21 dist-test-slave-zqpb krb5kdc[9045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:21.523041 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:33519
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:44325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:33519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:21.600385 9058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:21.600576 9058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:21.600600 9058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:21.601720 9058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:21.601763 9058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:21.601775 9058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:21.601799 9058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:21.601814 9058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:21.603253 9058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:44325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:33519
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:33519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:21.603461 9058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:21.603653 9058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:21.607999 9063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:21.608067 9058 server_base.cc:1034] running on GCE node
W20260619 02:49:21.608050 9066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:21.608052 9064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:21.608402 9058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:21.608680 9058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:21.609879 9058 hybrid_clock.cc:648] HybridClock initialized: now 1781837361609859 us; error 31 us; skew 500 ppm
Jun 19 02:49:21 dist-test-slave-zqpb krb5kdc[9045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837361, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:21.615535 9058 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:21.615962 9058 webserver.cc:460] Webserver started at http://127.19.113.126:38827/ using document root <none> and password file <none>
I20260619 02:49:21.616174 9058 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:21.616222 9058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:21.616345 9058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:21.617412 9058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "1f86ce3d7df2420db585fe37d789e14c"
format_stamp: "Formatted at 2026-06-19 02:49:21 on dist-test-slave-zqpb"
server_key: "4870962c8360e4c30c1971f2adabe5ea"
server_key_iv: "03a49351919a3f49846e4b57786e48ea"
server_key_version: "encryptionkey@0"
I20260619 02:49:21.617738 9058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "1f86ce3d7df2420db585fe37d789e14c"
format_stamp: "Formatted at 2026-06-19 02:49:21 on dist-test-slave-zqpb"
server_key: "4870962c8360e4c30c1971f2adabe5ea"
server_key_iv: "03a49351919a3f49846e4b57786e48ea"
server_key_version: "encryptionkey@0"
I20260619 02:49:21.619035 9058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:49:21.619710 9073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:21.619863 9058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:21.619953 9058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "1f86ce3d7df2420db585fe37d789e14c"
format_stamp: "Formatted at 2026-06-19 02:49:21 on dist-test-slave-zqpb"
server_key: "4870962c8360e4c30c1971f2adabe5ea"
server_key_iv: "03a49351919a3f49846e4b57786e48ea"
server_key_version: "encryptionkey@0"
I20260619 02:49:21.620015 9058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:21.630132 9058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:21.631752 9058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:21.631876 9058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:21.635488 9058 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:33519
I20260619 02:49:21.635519 9125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:33519 every 8 connection(s)
I20260619 02:49:21.635818 9058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:21.636590 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:21.637019 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9058
I20260619 02:49:21.637128 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:21.637290 19909 external_mini_cluster.cc:1428] Setting key 625abc06a94acee926335bd88781cfc0
I20260619 02:49:21.639323 9126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c: Bootstrap starting.
I20260619 02:49:21.639956 9126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:21.640249 9126 log.cc:824] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:21.640844 9126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c: No bootstrap required, opened a new log
I20260619 02:49:21.642783 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:21.637763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50019 (local address 127.19.113.126:33519)
0619 02:49:21.637939 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:21.637948 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:21.637980 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:49:21.638462 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:21.638470 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:21.638538 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:21.638681 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:21.638824 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:21.640112 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:21.640863 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:21.641437 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:21.641549 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:21.642119 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:21.642124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:21.642124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:21.642319 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:21.642355 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:21.642370 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:21.642455 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:21.642532 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:49:21.642560 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260619 02:49:21.643131 9126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f86ce3d7df2420db585fe37d789e14c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33519 } }
I20260619 02:49:21.643285 9126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:21.643339 9126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f86ce3d7df2420db585fe37d789e14c, State: Initialized, Role: FOLLOWER
I20260619 02:49:21.643438 9126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f86ce3d7df2420db585fe37d789e14c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33519 } }
I20260619 02:49:21.643510 9126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:21.643548 9126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:21.643594 9126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:21.643824 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:21.644127 9126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f86ce3d7df2420db585fe37d789e14c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33519 } }
I20260619 02:49:21.644212 9126 leader_election.cc:302] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f86ce3d7df2420db585fe37d789e14c; no voters:
I20260619 02:49:21.644349 9126 leader_election.cc:288] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:21.644415 9131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:21.644567 9126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:21.644606 9131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [term 1 LEADER]: Becoming Leader. State: Replica: 1f86ce3d7df2420db585fe37d789e14c, State: Running, Role: LEADER
I20260619 02:49:21.644766 9131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f86ce3d7df2420db585fe37d789e14c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33519 } }
I20260619 02:49:21.645126 9133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f86ce3d7df2420db585fe37d789e14c. Latest consensus state: current_term: 1 leader_uuid: "1f86ce3d7df2420db585fe37d789e14c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f86ce3d7df2420db585fe37d789e14c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33519 } } }
I20260619 02:49:21.645202 9133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:21.645344 9132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f86ce3d7df2420db585fe37d789e14c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f86ce3d7df2420db585fe37d789e14c" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 33519 } } }
I20260619 02:49:21.645406 9132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:21.645635 9144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:21.646104 9144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:21.647583 9144 catalog_manager.cc:1347] Generated new cluster ID: b7c55efdefed4c2886a484b03e650e2e
I20260619 02:49:21.647634 9144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:21.649044 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:21.645183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44127 (local address 127.19.113.126:33519)
0619 02:49:21.645216 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:21.645217 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:21.645914 (+ 697us) server_negotiation.cc:407] Connection header received
0619 02:49:21.645989 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:21.645990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:21.645999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:21.646055 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:21.646206 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:21.646850 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:21.647769 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:21.648090 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:21.648124 (+ 34us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:21.648815 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:21.648815 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:21.648816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:21.648820 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:21.648831 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:21.648834 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:21.648867 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:21.648969 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:49:21.648978 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:49:21.656454 9144 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:21.656948 9144 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:21.667764 9144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f86ce3d7df2420db585fe37d789e14c: Generated new TSK 0
I20260619 02:49:21.667927 9144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:21.672225 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9058
W20260619 02:49:21.675874 9043 connection.cc:535] client connection to 127.19.113.126:33519 recv error: Network error: failed to read from TLS socket (remote: 127.19.113.126:33519): Connection reset by peer (error 104)
2026-06-19T02:49:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:21 dist-test-slave-zqpb krb5kdc[9157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:21 dist-test-slave-zqpb krb5kdc[9157](info): set up 2 sockets
Jun 19 02:49:21 dist-test-slave-zqpb krb5kdc[9157](info): commencing operation
krb5kdc: starting...
W20260619 02:49:22.614753 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:22 dist-test-slave-zqpb krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:22.724956 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:41817
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:41817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:22.802263 9170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:22.802452 9170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:22.802475 9170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:22.803619 9170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:22.803658 9170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:22.803671 9170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:22.803682 9170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:22.803694 9170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:22.805097 9170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:41817
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:41817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:22.805303 9170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:22.805503 9170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:22.810070 9176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:22.810072 9175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:22.810155 9178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:22.810194 9170 server_base.cc:1034] running on GCE node
I20260619 02:49:22.810407 9170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:22.810637 9170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:22.811762 9170 hybrid_clock.cc:648] HybridClock initialized: now 1781837362811748 us; error 29 us; skew 500 ppm
Jun 19 02:49:22 dist-test-slave-zqpb krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837362, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:22.813982 9170 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:22.814348 9170 webserver.cc:460] Webserver started at http://127.19.113.126:34703/ using document root <none> and password file <none>
I20260619 02:49:22.814530 9170 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:22.814570 9170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:22.814657 9170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:22.815659 9170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "26c1f5938cd440cca4e909fd00e0f48a"
format_stamp: "Formatted at 2026-06-19 02:49:22 on dist-test-slave-zqpb"
server_key: "04a37d601c1e0e87a54c99dbeca00551"
server_key_iv: "ccddd557dea4296abd27fbe0292303c4"
server_key_version: "encryptionkey@0"
I20260619 02:49:22.815960 9170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "26c1f5938cd440cca4e909fd00e0f48a"
format_stamp: "Formatted at 2026-06-19 02:49:22 on dist-test-slave-zqpb"
server_key: "04a37d601c1e0e87a54c99dbeca00551"
server_key_iv: "ccddd557dea4296abd27fbe0292303c4"
server_key_version: "encryptionkey@0"
I20260619 02:49:22.817128 9170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:49:22.817785 9185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:22.817925 9170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:49:22.818006 9170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "26c1f5938cd440cca4e909fd00e0f48a"
format_stamp: "Formatted at 2026-06-19 02:49:22 on dist-test-slave-zqpb"
server_key: "04a37d601c1e0e87a54c99dbeca00551"
server_key_iv: "ccddd557dea4296abd27fbe0292303c4"
server_key_version: "encryptionkey@0"
I20260619 02:49:22.818077 9170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:22.821195 9170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:22.822772 9170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:22.822917 9170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:22.826380 9170 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:41817
I20260619 02:49:22.826413 9237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:41817 every 8 connection(s)
I20260619 02:49:22.826690 9170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:22.827611 9238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:22.828909 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9170
I20260619 02:49:22.829008 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:22.829150 19909 external_mini_cluster.cc:1428] Setting key 2e89574a363424ad8f66b3f1c68a2f7b
I20260619 02:49:22.829972 9238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a: Bootstrap starting.
I20260619 02:49:22.830631 9238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:22.831000 9238 log.cc:824] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:22.831481 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:22.829586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48225 (local address 127.19.113.126:41817)
0619 02:49:22.829766 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:22.829775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:22.829806 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:49:22.830322 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:22.830332 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:22.830366 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:22.830526 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:22.831075 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:22.831079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:22.831079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:22.831083 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:22.831121 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:22.831136 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:22.831221 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:22.831257 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 02:49:22.831287 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260619 02:49:22.831681 9238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a: No bootstrap required, opened a new log
I20260619 02:49:22.832370 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:22.833578 9238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c1f5938cd440cca4e909fd00e0f48a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41817 } }
I20260619 02:49:22.833699 9238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:22.833729 9238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c1f5938cd440cca4e909fd00e0f48a, State: Initialized, Role: FOLLOWER
I20260619 02:49:22.833837 9238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c1f5938cd440cca4e909fd00e0f48a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41817 } }
I20260619 02:49:22.833901 9238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:22.833940 9238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:22.833984 9238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:22.834450 9238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c1f5938cd440cca4e909fd00e0f48a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41817 } }
I20260619 02:49:22.834538 9238 leader_election.cc:302] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c1f5938cd440cca4e909fd00e0f48a; no voters:
I20260619 02:49:22.834683 9238 leader_election.cc:288] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:22.834710 9247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:22.834846 9247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [term 1 LEADER]: Becoming Leader. State: Replica: 26c1f5938cd440cca4e909fd00e0f48a, State: Running, Role: LEADER
I20260619 02:49:22.834923 9238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:22.834991 9247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c1f5938cd440cca4e909fd00e0f48a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41817 } }
I20260619 02:49:22.835367 9249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c1f5938cd440cca4e909fd00e0f48a. Latest consensus state: current_term: 1 leader_uuid: "26c1f5938cd440cca4e909fd00e0f48a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c1f5938cd440cca4e909fd00e0f48a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41817 } } }
I20260619 02:49:22.835451 9249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:22.835630 9248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c1f5938cd440cca4e909fd00e0f48a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c1f5938cd440cca4e909fd00e0f48a" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41817 } } }
I20260619 02:49:22.835981 9248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:22.836329 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:22.835165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39287 (local address 127.19.113.126:41817)
0619 02:49:22.835196 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:22.835198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:22.835333 (+ 135us) server_negotiation.cc:407] Connection header received
0619 02:49:22.835423 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:22.835424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:22.835428 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:22.835454 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:22.836077 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:22.836078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:22.836078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:22.836082 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:22.836096 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:22.836100 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:22.836144 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:22.836260 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 02:49:22.836273 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 02:49:22.836596 9264 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:22.836649 9264 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:22.836689 9257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:22.836833 9257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:22.838106 9257 catalog_manager.cc:1347] Generated new cluster ID: 6a3eab53b313494ebbccd572f03a3288
I20260619 02:49:22.838155 9257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:22.841861 9257 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:22.842003 9257 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:22.851627 9257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26c1f5938cd440cca4e909fd00e0f48a: Generated new TSK 0
I20260619 02:49:22.851776 9257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:22.854060 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9170
2026-06-19T02:49:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1183 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:22 dist-test-slave-zqpb krb5kdc[9269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:22 dist-test-slave-zqpb krb5kdc[9269](info): set up 2 sockets
Jun 19 02:49:22 dist-test-slave-zqpb krb5kdc[9269](info): commencing operation
krb5kdc: starting...
W20260619 02:49:23.810694 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:23 dist-test-slave-zqpb krb5kdc[9269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:23.917441 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45411
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:36941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:45411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:23.995218 9282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:23.995425 9282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:23.995446 9282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:23.996552 9282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:23.996644 9282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:23.996699 9282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:23.996723 9282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:23.996739 9282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:23.998229 9282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:36941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:45411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:23.998420 9282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:23.998597 9282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:24.003044 9290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:24.003178 9282 server_base.cc:1034] running on GCE node
W20260619 02:49:24.003065 9287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:24.003203 9288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:24.003486 9282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:24.003723 9282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:24.004854 9282 hybrid_clock.cc:648] HybridClock initialized: now 1781837364004843 us; error 28 us; skew 500 ppm
Jun 19 02:49:24 dist-test-slave-zqpb krb5kdc[9269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:24.007161 9282 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:24.007535 9282 webserver.cc:460] Webserver started at http://127.19.113.126:42895/ using document root <none> and password file <none>
I20260619 02:49:24.007727 9282 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:24.007772 9282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:24.007907 9282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:24.008888 9282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "ac71e6d2c39845608307284f96fa6ba7"
format_stamp: "Formatted at 2026-06-19 02:49:24 on dist-test-slave-zqpb"
server_key: "0bf49a74458f524ddd3d6548ab937cc4"
server_key_iv: "f8201265cabf31bf8eb5baf5b4aa52ee"
server_key_version: "encryptionkey@0"
I20260619 02:49:24.009215 9282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "ac71e6d2c39845608307284f96fa6ba7"
format_stamp: "Formatted at 2026-06-19 02:49:24 on dist-test-slave-zqpb"
server_key: "0bf49a74458f524ddd3d6548ab937cc4"
server_key_iv: "f8201265cabf31bf8eb5baf5b4aa52ee"
server_key_version: "encryptionkey@0"
I20260619 02:49:24.010380 9282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:49:24.011036 9297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:24.011183 9282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:24.011260 9282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "ac71e6d2c39845608307284f96fa6ba7"
format_stamp: "Formatted at 2026-06-19 02:49:24 on dist-test-slave-zqpb"
server_key: "0bf49a74458f524ddd3d6548ab937cc4"
server_key_iv: "f8201265cabf31bf8eb5baf5b4aa52ee"
server_key_version: "encryptionkey@0"
I20260619 02:49:24.011325 9282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:24.030198 9282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:24.032223 9282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:24.032356 9282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:24.035861 9282 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:45411
I20260619 02:49:24.035907 9349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:45411 every 8 connection(s)
I20260619 02:49:24.036226 9282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:24.036984 9350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:24.039131 9350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7: Bootstrap starting.
I20260619 02:49:24.039623 9350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:24.039834 9350 log.cc:824] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:24.040405 9350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7: No bootstrap required, opened a new log
I20260619 02:49:24.041138 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9282
I20260619 02:49:24.041250 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:24.041386 19909 external_mini_cluster.cc:1428] Setting key 21deb05e6fa57867f7174f6281b956ee
I20260619 02:49:24.042207 9350 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac71e6d2c39845608307284f96fa6ba7" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45411 } }
I20260619 02:49:24.042371 9350 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:24.042443 9350 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac71e6d2c39845608307284f96fa6ba7, State: Initialized, Role: FOLLOWER
I20260619 02:49:24.042578 9350 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac71e6d2c39845608307284f96fa6ba7" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45411 } }
I20260619 02:49:24.042659 9350 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:24.042681 9350 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:24.042726 9350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:24.043733 9350 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac71e6d2c39845608307284f96fa6ba7" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45411 } }
I20260619 02:49:24.043838 9350 leader_election.cc:302] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac71e6d2c39845608307284f96fa6ba7; no voters:
I20260619 02:49:24.043987 9350 leader_election.cc:288] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:24.044030 9355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:24.044159 9355 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [term 1 LEADER]: Becoming Leader. State: Replica: ac71e6d2c39845608307284f96fa6ba7, State: Running, Role: LEADER
I20260619 02:49:24.044210 9350 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:24.044289 9355 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac71e6d2c39845608307284f96fa6ba7" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45411 } }
I20260619 02:49:24.044637 9355 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac71e6d2c39845608307284f96fa6ba7. Latest consensus state: current_term: 1 leader_uuid: "ac71e6d2c39845608307284f96fa6ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac71e6d2c39845608307284f96fa6ba7" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45411 } } }
I20260619 02:49:24.044833 9355 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:24.044775 9357 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac71e6d2c39845608307284f96fa6ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac71e6d2c39845608307284f96fa6ba7" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45411 } } }
I20260619 02:49:24.045058 9357 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7 [sys.catalog]: This master's current role is: LEADER
W20260619 02:49:24.045859 9371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:24.045913 9371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:24.045966 9364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:24.046100 9364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:24.047644 9364 catalog_manager.cc:1347] Generated new cluster ID: 0899da1d657f40cab1e7de679c5ab2b9
I20260619 02:49:24.047703 9364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:49:24 dist-test-slave-zqpb krb5kdc[9269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:24.051389 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:24.041854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33039 (local address 127.19.113.126:45411)
0619 02:49:24.042078 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:24.042088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:24.042123 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:49:24.042234 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:24.042241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:24.042295 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:24.042437 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:24.043103 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:24.044447 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:24.045471 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:24.045968 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:24.046065 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:24.048452 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:24.048456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:24.048456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:24.048468 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:24.049947 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:24.050269 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:24.050272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:24.050273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:24.050374 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:24.050552 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:24.050553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:24.050553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:24.050790 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:24.050967 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:24.051069 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:49:24.051234 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":112,"threads_started":1}
I20260619 02:49:24.052268 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:24.060214 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:24.053491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43931 (local address 127.19.113.126:45411)
0619 02:49:24.053524 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:24.053525 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:24.055040 (+ 1515us) server_negotiation.cc:407] Connection header received
0619 02:49:24.055080 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:24.055081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:24.055092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:24.055135 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:24.055593 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:24.056072 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:24.056757 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:24.056965 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:24.057017 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:24.058030 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:24.058039 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:24.058039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:24.058046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:24.059275 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:24.059562 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:24.059563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:24.059564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:24.059618 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:24.059772 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:24.059773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:24.059773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:24.059873 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:24.059968 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:24.060061 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:49:24.060129 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:49:24.077445 9364 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:24.077940 9364 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:24.083242 9364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac71e6d2c39845608307284f96fa6ba7: Generated new TSK 0
I20260619 02:49:24.083421 9364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:24.086177 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9282
2026-06-19T02:49:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1230 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:24 dist-test-slave-zqpb krb5kdc[9381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:24 dist-test-slave-zqpb krb5kdc[9381](info): set up 2 sockets
Jun 19 02:49:24 dist-test-slave-zqpb krb5kdc[9381](info): commencing operation
krb5kdc: starting...
W20260619 02:49:25.058777 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:25 dist-test-slave-zqpb krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:25.171046 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:41925
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:41925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:25.249209 9394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:25.249404 9394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:25.249426 9394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:25.250537 9394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:25.250572 9394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:25.250584 9394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:25.250594 9394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:25.250605 9394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:25.252106 9394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:41925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:41925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:25.252308 9394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:25.252475 9394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:25.257093 9402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:25.257093 9399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:25.257288 9394 server_base.cc:1034] running on GCE node
W20260619 02:49:25.257167 9400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:25.257611 9394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:25.257921 9394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:25.259063 9394 hybrid_clock.cc:648] HybridClock initialized: now 1781837365259049 us; error 28 us; skew 500 ppm
Jun 19 02:49:25 dist-test-slave-zqpb krb5kdc[9381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837365, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:25.261476 9394 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:25.261930 9394 webserver.cc:460] Webserver started at http://127.19.113.126:44583/ using document root <none> and password file <none>
I20260619 02:49:25.262146 9394 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:25.262198 9394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:25.262326 9394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:25.263408 9394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "417b3b29214e46a6a31ff02abd91b227"
format_stamp: "Formatted at 2026-06-19 02:49:25 on dist-test-slave-zqpb"
server_key: "2a8d915def785c4bfd167e69af854504"
server_key_iv: "49331bd2bce1cc02a4008dde75790029"
server_key_version: "encryptionkey@0"
I20260619 02:49:25.263736 9394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "417b3b29214e46a6a31ff02abd91b227"
format_stamp: "Formatted at 2026-06-19 02:49:25 on dist-test-slave-zqpb"
server_key: "2a8d915def785c4bfd167e69af854504"
server_key_iv: "49331bd2bce1cc02a4008dde75790029"
server_key_version: "encryptionkey@0"
I20260619 02:49:25.264937 9394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:25.265545 9409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:25.265689 9394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:25.265769 9394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "417b3b29214e46a6a31ff02abd91b227"
format_stamp: "Formatted at 2026-06-19 02:49:25 on dist-test-slave-zqpb"
server_key: "2a8d915def785c4bfd167e69af854504"
server_key_iv: "49331bd2bce1cc02a4008dde75790029"
server_key_version: "encryptionkey@0"
I20260619 02:49:25.265862 9394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:25.276160 9394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:25.277779 9394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:25.277909 9394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:25.281459 9394 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:41925
I20260619 02:49:25.281509 9461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:41925 every 8 connection(s)
I20260619 02:49:25.281817 9394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:25.282651 9462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:25.284854 9462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227: Bootstrap starting.
I20260619 02:49:25.285157 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9394
I20260619 02:49:25.285266 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:25.285406 9462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:25.285430 19909 external_mini_cluster.cc:1428] Setting key 00a7bb77c5527661d73c544385af6f2e
I20260619 02:49:25.285691 9462 log.cc:824] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:25.286464 9462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227: No bootstrap required, opened a new log
I20260619 02:49:25.288914 9462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417b3b29214e46a6a31ff02abd91b227" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41925 } }
I20260619 02:49:25.289049 9462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:25.289080 9462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417b3b29214e46a6a31ff02abd91b227, State: Initialized, Role: FOLLOWER
I20260619 02:49:25.289213 9462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417b3b29214e46a6a31ff02abd91b227" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41925 } }
I20260619 02:49:25.289289 9462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:25.289323 9462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:25.289376 9462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:25.290086 9462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417b3b29214e46a6a31ff02abd91b227" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41925 } }
I20260619 02:49:25.290196 9462 leader_election.cc:302] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 417b3b29214e46a6a31ff02abd91b227; no voters:
I20260619 02:49:25.290403 9462 leader_election.cc:288] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:25.290458 9467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:25.290597 9467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [term 1 LEADER]: Becoming Leader. State: Replica: 417b3b29214e46a6a31ff02abd91b227, State: Running, Role: LEADER
I20260619 02:49:25.290632 9462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:25.290688 9467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417b3b29214e46a6a31ff02abd91b227" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41925 } }
I20260619 02:49:25.291050 9469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 417b3b29214e46a6a31ff02abd91b227. Latest consensus state: current_term: 1 leader_uuid: "417b3b29214e46a6a31ff02abd91b227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417b3b29214e46a6a31ff02abd91b227" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41925 } } }
I20260619 02:49:25.291272 9469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:25.291206 9468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "417b3b29214e46a6a31ff02abd91b227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417b3b29214e46a6a31ff02abd91b227" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41925 } } }
I20260619 02:49:25.291528 9468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:25.291788 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:25.285912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47897 (local address 127.19.113.126:41925)
0619 02:49:25.286072 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:25.286081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:25.286107 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:49:25.286686 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:25.286695 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:25.286756 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:25.286940 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:25.287182 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:25.288610 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:25.289488 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:25.290075 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:25.290198 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:25.291049 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:25.291054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:25.291054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:25.291264 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:25.291302 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:25.291318 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:25.291360 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:25.291496 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 02:49:25.291567 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260619 02:49:25.291582 9476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:25.292598 9476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:25.292682 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:25.294155 9476 catalog_manager.cc:1347] Generated new cluster ID: 3823bb0390cd4772a9e2cf4f6476de80
I20260619 02:49:25.294202 9476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:25.297762 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:25.294139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58671 (local address 127.19.113.126:41925)
0619 02:49:25.294183 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:25.294185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:25.294418 (+ 233us) server_negotiation.cc:407] Connection header received
0619 02:49:25.294449 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:25.294450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:25.294462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:25.294525 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:25.294650 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:25.295386 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:25.296462 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:25.296759 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:25.296827 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:25.297540 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:25.297541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:25.297541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:25.297545 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:25.297560 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:25.297565 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:25.297621 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:25.297664 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 02:49:25.297687 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:49:25.309067 9476 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:25.309564 9476 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:25.320108 9476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 417b3b29214e46a6a31ff02abd91b227: Generated new TSK 0
I20260619 02:49:25.320272 9476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:25.328433 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9394
2026-06-19T02:49:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:25 dist-test-slave-zqpb krb5kdc[9493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:25 dist-test-slave-zqpb krb5kdc[9493](info): set up 2 sockets
Jun 19 02:49:25 dist-test-slave-zqpb krb5kdc[9493](info): commencing operation
krb5kdc: starting...
W20260619 02:49:26.282778 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:26 dist-test-slave-zqpb krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:26.393116 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45561
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:45561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:26.471288 9506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:26.471510 9506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:26.471542 9506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:26.472685 9506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:26.472725 9506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:26.472743 9506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:26.472759 9506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:26.472775 9506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:26.474227 9506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:45561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:45561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:26.474479 9506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:26.474682 9506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:26.479365 9511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:26.479354 9512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:26.479357 9514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:26.479418 9506 server_base.cc:1034] running on GCE node
I20260619 02:49:26.479743 9506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:26.480026 9506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:26.481177 9506 hybrid_clock.cc:648] HybridClock initialized: now 1781837366481160 us; error 28 us; skew 500 ppm
Jun 19 02:49:26 dist-test-slave-zqpb krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837366, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:26.483632 9506 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:26.484030 9506 webserver.cc:460] Webserver started at http://127.19.113.126:45177/ using document root <none> and password file <none>
I20260619 02:49:26.484220 9506 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:26.484264 9506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:26.484392 9506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:26.485427 9506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "857ce502e4544eaba6ad78ffc52aa57d"
format_stamp: "Formatted at 2026-06-19 02:49:26 on dist-test-slave-zqpb"
server_key: "02beaf6d617707f02f1c793f481085a2"
server_key_iv: "7d10e01039a0d088779d9a2938847569"
server_key_version: "encryptionkey@0"
I20260619 02:49:26.485749 9506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "857ce502e4544eaba6ad78ffc52aa57d"
format_stamp: "Formatted at 2026-06-19 02:49:26 on dist-test-slave-zqpb"
server_key: "02beaf6d617707f02f1c793f481085a2"
server_key_iv: "7d10e01039a0d088779d9a2938847569"
server_key_version: "encryptionkey@0"
I20260619 02:49:26.486995 9506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:26.487704 9521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:26.487854 9506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:26.487936 9506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "857ce502e4544eaba6ad78ffc52aa57d"
format_stamp: "Formatted at 2026-06-19 02:49:26 on dist-test-slave-zqpb"
server_key: "02beaf6d617707f02f1c793f481085a2"
server_key_iv: "7d10e01039a0d088779d9a2938847569"
server_key_version: "encryptionkey@0"
I20260619 02:49:26.488000 9506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:26.491197 9506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:26.493180 9506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:26.493319 9506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:26.497200 9506 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:45561
I20260619 02:49:26.497229 9573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:45561 every 8 connection(s)
I20260619 02:49:26.497526 9506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:26.498423 9574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:26.500684 9574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d: Bootstrap starting.
I20260619 02:49:26.501187 9574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:26.501410 9574 log.cc:824] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:26.502010 9574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d: No bootstrap required, opened a new log
I20260619 02:49:26.503652 9574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857ce502e4544eaba6ad78ffc52aa57d" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45561 } }
I20260619 02:49:26.503785 9574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:49:26.503805 9574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857ce502e4544eaba6ad78ffc52aa57d, State: Initialized, Role: FOLLOWER
I20260619 02:49:26.503906 9574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857ce502e4544eaba6ad78ffc52aa57d" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45561 } }
I20260619 02:49:26.503975 9574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:49:26.503996 9574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:49:26.504029 9574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:26.504767 9574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857ce502e4544eaba6ad78ffc52aa57d" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45561 } }
I20260619 02:49:26.504851 9574 leader_election.cc:302] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 857ce502e4544eaba6ad78ffc52aa57d; no voters:
I20260619 02:49:26.504971 9574 leader_election.cc:288] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:49:26.505024 9577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:26.505192 9577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [term 1 LEADER]: Becoming Leader. State: Replica: 857ce502e4544eaba6ad78ffc52aa57d, State: Running, Role: LEADER
I20260619 02:49:26.505178 9574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:26.505316 9577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857ce502e4544eaba6ad78ffc52aa57d" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45561 } }
I20260619 02:49:26.505645 9579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 857ce502e4544eaba6ad78ffc52aa57d. Latest consensus state: current_term: 1 leader_uuid: "857ce502e4544eaba6ad78ffc52aa57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857ce502e4544eaba6ad78ffc52aa57d" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45561 } } }
I20260619 02:49:26.505645 9578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "857ce502e4544eaba6ad78ffc52aa57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857ce502e4544eaba6ad78ffc52aa57d" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 45561 } } }
I20260619 02:49:26.505810 9578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:26.505816 9579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:26.506052 9586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:26.506561 9586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:26.507169 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9506
I20260619 02:49:26.507272 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:26.507431 19909 external_mini_cluster.cc:1428] Setting key 289485474b5d2dda05365315623aaf88
I20260619 02:49:26.508059 9586 catalog_manager.cc:1347] Generated new cluster ID: 6bca64bbfaae465e980ce2ac78d64953
I20260619 02:49:26.508106 9586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:26.510190 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:26.507854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37671 (local address 127.19.113.126:45561)
0619 02:49:26.508124 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:26.508132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:26.508379 (+ 247us) server_negotiation.cc:407] Connection header received
0619 02:49:26.508504 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:26.508512 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:26.508541 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:26.508658 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:26.509663 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:26.509667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:26.509668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:26.509672 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:26.509712 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:26.509726 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:26.509795 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:26.509966 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 02:49:26.510009 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":69,"threads_started":1}
I20260619 02:49:26.511061 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:26.513413 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:26.512273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52185 (local address 127.19.113.126:45561)
0619 02:49:26.512308 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:26.512310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:26.512501 (+ 191us) server_negotiation.cc:407] Connection header received
0619 02:49:26.512547 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:26.512548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:26.512553 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:26.512614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:26.513187 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:26.513188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:26.513189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:49:26.513194 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:26.513214 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:49:26.513218 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:26.513282 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:26.513344 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:49:26.513358 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:49:26.517246 9586 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:26.517383 9586 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:26.525782 9586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 857ce502e4544eaba6ad78ffc52aa57d: Generated new TSK 0
I20260619 02:49:26.525969 9586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:49:26.530821 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9506
2026-06-19T02:49:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1202 ms)
[----------] 18 tests from AuthTokenIssuingTest (21840 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:26 dist-test-slave-zqpb krb5kdc[9605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:26 dist-test-slave-zqpb krb5kdc[9605](info): set up 2 sockets
Jun 19 02:49:26 dist-test-slave-zqpb krb5kdc[9605](info): commencing operation
krb5kdc: starting...
W20260619 02:49:27.490903 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:27 dist-test-slave-zqpb krb5kdc[9605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:27.599654 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:39757
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:39757,127.19.113.125:38641,127.19.113.124:46579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:27.677549 9618 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:49:27.677739 9618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:27.677754 9618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:27.677774 9618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:27.678936 9618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:27.678973 9618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:27.678985 9618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:27.678996 9618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:27.679006 9618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:27.680408 9618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:39757,127.19.113.125:38641,127.19.113.124:46579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:39757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:27.680617 9618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:27.680821 9618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:27.685286 9626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:27.685298 9624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:27.685292 9623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:27.685412 9618 server_base.cc:1034] running on GCE node
I20260619 02:49:27.685851 9618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:27.686112 9618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:27.687250 9618 hybrid_clock.cc:648] HybridClock initialized: now 1781837367687239 us; error 27 us; skew 500 ppm
Jun 19 02:49:27 dist-test-slave-zqpb krb5kdc[9605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:27.693471 9618 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:27.693856 9618 webserver.cc:460] Webserver started at http://127.19.113.126:44471/ using document root <none> and password file <none>
I20260619 02:49:27.694059 9618 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:27.694105 9618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:27.694224 9618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:27.695276 9618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "7e4b4aee7ac947499c7b351c620dca31"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "9283200698e003459f22544adb639b90"
server_key_iv: "8c1ba683b53c759eeae51638d4a01313"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.695583 9618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "7e4b4aee7ac947499c7b351c620dca31"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "9283200698e003459f22544adb639b90"
server_key_iv: "8c1ba683b53c759eeae51638d4a01313"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.696789 9618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:27.697499 9633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:27.697665 9618 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:27.697743 9618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "7e4b4aee7ac947499c7b351c620dca31"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "9283200698e003459f22544adb639b90"
server_key_iv: "8c1ba683b53c759eeae51638d4a01313"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.697808 9618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:27.719422 9618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:27.721246 9618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:27.721376 9618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:27.724968 9618 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:39757
I20260619 02:49:27.725018 9685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:39757 every 8 connection(s)
I20260619 02:49:27.725307 9618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:27.726078 9686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:27.727550 9686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:49:27 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:27.733839 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9618
I20260619 02:49:27.733958 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:27.734129 19909 external_mini_cluster.cc:1428] Setting key b8a90a2cb2ca296fb5087e60f149b1ba
I20260619 02:49:27.736426 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.728220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.19.113.126:39757)
0619 02:49:27.728377 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:27.728386 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:27.728500 (+ 114us) server_negotiation.cc:407] Connection header received
0619 02:49:27.728983 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:27.728985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:27.729003 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:27.729086 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:27.729558 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.730367 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.731268 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.731578 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.731664 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:27.733489 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:27.733495 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:27.733495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:27.733501 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:27.735079 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:27.735395 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:27.735397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:27.735398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:27.735498 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:27.735725 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:27.735726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:27.735726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:27.735921 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:27.735978 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:27.736173 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:49:27.736256 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260619 02:49:27.736443 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.728194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:39757 (local address 127.0.0.1:48496)
0619 02:49:27.728340 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.728377 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:27.728844 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:27.729096 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:27.729098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:27.729333 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:27.729484 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:27.729489 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.730454 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.730455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:27.731183 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:27.731185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.731647 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.731647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:27.731696 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:27.732138 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:27.732146 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:27.733416 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:27.735147 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:27.735153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:27.735155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:27.735329 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:27.735564 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:27.735565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:27.735566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:27.735668 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:27.735976 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:27.735982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:27.736050 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 02:49:27.736141 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:49:27.736328 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":45,"thread_start_us":65,"threads_started":1}
I20260619 02:49:27.737072 9686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:27.737350 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.737250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:34720)
0619 02:49:27.737281 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.737328 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260619 02:49:27.737499 9636 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
I20260619 02:49:27.737907 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.737803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:34728)
0619 02:49:27.737848 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.737885 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:49:27.738005 9686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:38641: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.19.113.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab.
Entry for principal kudu/127.19.113.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab.
I20260619 02:49:27.755873 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.125:38641
--webserver_interface=127.19.113.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab
--principal=kudu/127.19.113.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:39757,127.19.113.125:38641,127.19.113.124:46579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:49:27.787220 9686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } attempt: 1
I20260619 02:49:27.787732 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.787597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:49098)
0619 02:49:27.787652 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.787697 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:49:27.788075 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.788001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:49114)
0619 02:49:27.788035 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.788058 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 02:49:27.788164 9686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:38641: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
W20260619 02:49:27.834990 9693 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:49:27.835201 9693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:27.835216 9693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:27.835235 9693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:27.836381 9693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:27.836416 9693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:27.836427 9693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:27.836439 9693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:27.836449 9693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:27.837868 9693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:39757,127.19.113.125:38641,127.19.113.124:46579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab
--principal=kudu/127.19.113.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.125:38641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.113.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:27.838065 9693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:27.838227 9693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:49:27.842895 9693 server_base.cc:1034] running on GCE node
W20260619 02:49:27.842819 9701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:27.842794 9699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:27.842814 9698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:27.843237 9693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:27.843505 9693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:27.844642 9693 hybrid_clock.cc:648] HybridClock initialized: now 1781837367844626 us; error 33 us; skew 500 ppm
Jun 19 02:49:27 dist-test-slave-zqpb krb5kdc[9605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:27.846812 9693 init.cc:377] Logged in from keytab as kudu/127.19.113.125@KRBTEST.COM (short username kudu)
I20260619 02:49:27.847239 9693 webserver.cc:460] Webserver started at http://127.19.113.125:43957/ using document root <none> and password file <none>
I20260619 02:49:27.847447 9693 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:27.847491 9693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:27.847616 9693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:27.848628 9693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/data/instance:
uuid: "7559d21d76d94ee8acde492793bbd3ea"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "b67628e2cab68cc176ceb0844fe4bda2"
server_key_iv: "8adb6f1d8413818cd2e2729dcc22a42c"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.848937 9693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/wal/instance:
uuid: "7559d21d76d94ee8acde492793bbd3ea"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "b67628e2cab68cc176ceb0844fe4bda2"
server_key_iv: "8adb6f1d8413818cd2e2729dcc22a42c"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.850117 9693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:27.850787 9708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:27.850987 9693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:27.851069 9693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/wal
uuid: "7559d21d76d94ee8acde492793bbd3ea"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "b67628e2cab68cc176ceb0844fe4bda2"
server_key_iv: "8adb6f1d8413818cd2e2729dcc22a42c"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.851121 9693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:27.856288 9686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } attempt: 2
I20260619 02:49:27.856664 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.856546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:49128)
0619 02:49:27.856599 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.856636 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:49:27.856927 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.856864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:49140)
0619 02:49:27.856892 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.856913 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260619 02:49:27.856997 9686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:38641: Network error: Client connection negotiation failed: client connection to 127.19.113.125:38641: connect: Connection refused (error 111)
I20260619 02:49:27.861166 9693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:27.863183 9693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:27.863282 9693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:27.866643 9693 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.125:38641
I20260619 02:49:27.866680 9760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.125:38641 every 8 connection(s)
I20260619 02:49:27.867033 9693 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
I20260619 02:49:27.867873 9761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:27.869328 9761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:27.869828 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9693
I20260619 02:49:27.869938 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-1/wal/instance
I20260619 02:49:27.870141 19909 external_mini_cluster.cc:1428] Setting key 9c5c02c8e09ca6eb5ce49aae65ce9788
Jun 19 02:49:27 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.113.124@KRBTEST.COM; defaulting to no policy
I20260619 02:49:27.877884 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.870072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.19.113.126:39757)
0619 02:49:27.870123 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:27.870125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:27.870324 (+ 199us) server_negotiation.cc:407] Connection header received
0619 02:49:27.870823 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:27.870824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:27.870840 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:27.870905 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:27.871494 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.871997 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.872916 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.873162 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.873218 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:27.875407 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:27.875408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:27.875409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:27.875416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:27.876635 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:27.877060 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:27.877061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:27.877061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:27.877111 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:27.877336 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:27.877337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:27.877338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:27.877457 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:27.877520 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:27.877742 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 02:49:27.877812 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:49:27.878050 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.870032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:39757 (local address 127.0.0.1:54808)
0619 02:49:27.870189 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.870236 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:27.870710 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:27.870974 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:27.870977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:27.871205 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:27.871400 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:27.871404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.872082 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.872083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:27.872838 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:27.872840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.873236 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.873237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:27.873356 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:27.873856 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:27.873863 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:27.875323 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:27.876720 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:27.876728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:27.876730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:27.876999 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:27.877159 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:27.877160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:27.877160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:27.877252 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:27.877518 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:27.877523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:27.877633 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 02:49:27.877717 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:49:27.877827 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260619 02:49:27.878584 9761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.124@KRBTEST.COM" created.
Jun 19 02:49:27 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:49:27.886284 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.878808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:49148)
0619 02:49:27.878910 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.878920 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:27.878957 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:27.879206 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:27.879207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:27.879338 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:27.879391 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:27.879393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.880514 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.880515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:27.881246 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:27.881248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.881678 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.881679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:27.881750 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:27.882217 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:27.882222 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:27.883745 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:27.885271 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:27.885272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:27.885273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:27.885470 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:27.885642 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:27.885642 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:27.885643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:27.885702 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:27.885940 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:27.885941 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:27.885986 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 02:49:27.886062 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:49:27.886161 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 02:49:27.886313 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.878930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.19.113.125:38641)
0619 02:49:27.879069 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:27.879073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:27.879092 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:49:27.879111 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:27.879114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:27.879126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:27.879190 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:27.879456 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.880417 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.881316 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.881611 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.881661 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:27.883846 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:27.883853 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:27.883854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:27.883859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:27.885189 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:27.885541 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:27.885543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:27.885543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:27.885589 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:27.885761 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:27.885762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:27.885762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:27.885886 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:27.885942 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:27.886082 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:49:27.886206 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260619 02:49:27.886937 9761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:27.887246 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.887146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:60990)
0619 02:49:27.887184 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.887227 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 02:49:27.887390 9709 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
I20260619 02:49:27.887734 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.887649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:60996)
0619 02:49:27.887689 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.887713 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:49:27.887832 9761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:46579: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab.
Entry for principal kudu/127.19.113.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab.
I20260619 02:49:27.893497 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.124:46579
--webserver_interface=127.19.113.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab
--principal=kudu/127.19.113.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:39757,127.19.113.125:38641,127.19.113.124:46579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:49:27.937057 9761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } attempt: 1
I20260619 02:49:27.937563 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.937421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32772)
0619 02:49:27.937485 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.937530 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 02:49:27.937866 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.937780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32774)
0619 02:49:27.937821 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.937847 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 02:49:27.937970 9761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:46579: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
I20260619 02:49:27.948151 9686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } attempt: 3
Jun 19 02:49:27 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:49:27.955431 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.948620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.19.113.125:38641)
0619 02:49:27.948683 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:27.948685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:27.948697 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:49:27.948724 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:27.948726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:27.948738 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:27.948801 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:27.949082 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.949856 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.950526 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.950846 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.950943 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:27.952859 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:27.952860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:27.952860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:27.952866 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:27.954239 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:27.954686 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:27.954687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:27.954687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:27.954746 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:27.954977 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:27.954978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:27.954978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:27.955095 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:27.955143 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:27.955301 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 02:49:27.955365 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:49:27.955482 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.948527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:49162)
0619 02:49:27.948581 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.948595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:27.948654 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:27.948807 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:27.948808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:27.948955 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:27.949016 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:27.949017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.949943 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.949943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:27.950457 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:27.950459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:27.950961 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:27.950962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:27.951031 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:27.951479 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:27.951483 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:27.952758 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:27.954312 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:27.954313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:27.954313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:27.954527 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:27.954824 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:27.954824 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:27.954825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:27.954914 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:27.955153 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:27.955154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:27.955206 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:49:27.955285 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:49:27.955386 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:49:27.955871 9686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:27.956204 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.956099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32776)
0619 02:49:27.956143 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.956178 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 02:49:27.956604 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.956520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32792)
0619 02:49:27.956555 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.956583 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 02:49:27.956722 9686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:46579: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
W20260619 02:49:27.976137 9768 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:49:27.976339 9768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:27.976356 9768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:27.976374 9768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:27.977540 9768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:27.977578 9768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:27.977591 9768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:27.977602 9768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:27.977627 9768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:27.979183 9768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:39757,127.19.113.125:38641,127.19.113.124:46579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab
--principal=kudu/127.19.113.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.124:46579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.113.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:27.979414 9768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:27.979616 9768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:27.984117 9773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:27.984120 9774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:27.984319 9768 server_base.cc:1034] running on GCE node
W20260619 02:49:27.984423 9776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:27.984699 9768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:27.984990 9768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:27.986148 9768 hybrid_clock.cc:648] HybridClock initialized: now 1781837367986131 us; error 32 us; skew 500 ppm
I20260619 02:49:27.987857 9686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } attempt: 1
I20260619 02:49:27.988241 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.988118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32800)
0619 02:49:27.988176 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.988210 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
Jun 19 02:49:27 dist-test-slave-zqpb krb5kdc[9605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:27.988570 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:27.988507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32806)
0619 02:49:27.988531 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:27.988553 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 02:49:27.988667 9686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:46579: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
I20260619 02:49:27.988684 9768 init.cc:377] Logged in from keytab as kudu/127.19.113.124@KRBTEST.COM (short username kudu)
I20260619 02:49:27.989066 9768 webserver.cc:460] Webserver started at http://127.19.113.124:46199/ using document root <none> and password file <none>
I20260619 02:49:27.989249 9768 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:27.989287 9768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:27.989377 9768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:27.990365 9768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/data/instance:
uuid: "c586e8000eec4d639a4a7840e140edbe"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "9ceaf1a05fea215b5bc9683977f9da4f"
server_key_iv: "f12ce22b4e3a9a711030960a6301d7ae"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.990654 9768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/wal/instance:
uuid: "c586e8000eec4d639a4a7840e140edbe"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "9ceaf1a05fea215b5bc9683977f9da4f"
server_key_iv: "f12ce22b4e3a9a711030960a6301d7ae"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.991946 9768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:27.992589 9783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:27.992733 9768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:27.992810 9768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/wal
uuid: "c586e8000eec4d639a4a7840e140edbe"
format_stamp: "Formatted at 2026-06-19 02:49:27 on dist-test-slave-zqpb"
server_key: "9ceaf1a05fea215b5bc9683977f9da4f"
server_key_iv: "f12ce22b4e3a9a711030960a6301d7ae"
server_key_version: "encryptionkey@0"
I20260619 02:49:27.992871 9768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:28.006107 9761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } attempt: 2
I20260619 02:49:28.006536 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.006422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32820)
0619 02:49:28.006475 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:28.006505 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:49:28.006844 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.006777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32832)
0619 02:49:28.006807 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:28.006830 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260619 02:49:28.006937 9761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:46579: Network error: Client connection negotiation failed: client connection to 127.19.113.124:46579: connect: Connection refused (error 111)
I20260619 02:49:28.009919 9768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:28.011585 9768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:28.011687 9768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:28.015298 9768 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.124:46579
I20260619 02:49:28.015327 9835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.124:46579 every 8 connection(s)
I20260619 02:49:28.015640 9768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
I20260619 02:49:28.016479 9836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:28.017853 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9768
I20260619 02:49:28.017961 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837111403550-19909-0/minicluster-data/master-2/wal/instance
I20260619 02:49:28.018105 19909 external_mini_cluster.cc:1428] Setting key b6c0db8a75c00b7171e342135dd3f065
I20260619 02:49:28.017911 9836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:49:28 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
Jun 19 02:49:28 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:49:28.026415 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.018582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.19.113.126:39757)
0619 02:49:28.018625 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.018627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.018893 (+ 266us) server_negotiation.cc:407] Connection header received
0619 02:49:28.019346 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.019347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.019360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.019406 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.019897 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.020374 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.021188 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.021372 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.021425 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.023619 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.023620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.023621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.023628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.025044 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.025428 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.025429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.025430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.025482 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.025723 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.025724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.025724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.025853 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.025952 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.026234 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.026321 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:49:28.026633 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.018757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32842 (local address 127.19.113.124:46579)
0619 02:49:28.018903 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.018907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.018937 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:49:28.019267 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.019272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.019291 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.019374 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.019974 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.020754 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.021455 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.021768 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.021832 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.024076 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.024083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.024083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.024090 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.025512 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.025839 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.025843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.025843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.025900 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.026068 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.026068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.026069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.026193 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.026334 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.026410 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.026480 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260619 02:49:28.026638 9839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.018577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:39757 (local address 127.0.0.1:54812)
0619 02:49:28.018728 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:28.018761 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:28.019259 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:28.019422 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:28.019425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:28.019670 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:28.019816 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.019820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.020438 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.020439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.021111 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.021113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.021433 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.021435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.021554 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.022035 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:28.022044 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:28.023538 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:28.025114 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.025120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.025122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.025367 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.025531 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.025532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.025532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.025654 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.025959 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:28.025965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:28.026113 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 02:49:28.026192 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:49:28.026343 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260619 02:49:28.027128 9836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:49:28 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
Jun 19 02:49:28 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:28.033218 9839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.027362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:38641 (local address 127.0.0.1:49176)
0619 02:49:28.027479 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:28.027488 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:28.027531 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:28.027674 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:28.027675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:28.027807 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:28.027850 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.027851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.028420 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.028421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.028928 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.028929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.029258 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.029258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.029328 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.029668 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:28.029673 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:28.030796 (+ 1123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:28.032361 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.032362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.032363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.032512 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.032687 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.032688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.032688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.032736 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.032947 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:28.032948 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:28.032987 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:49:28.033040 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:49:28.033121 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260619 02:49:28.033218 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.027412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.19.113.125:38641)
0619 02:49:28.027454 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.027456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.027525 (+ 69us) server_negotiation.cc:407] Connection header received
0619 02:49:28.027611 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.027612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.027622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.027670 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.027913 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.028361 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.028998 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.029204 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.029255 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.030905 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.030907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.030907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.030914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.032267 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.032577 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.032578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.032579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.032637 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.032783 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.032784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.032784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.032897 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.032948 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.033066 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.033130 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:49:28.033811 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.027439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.19.113.126:39757)
0619 02:49:28.027475 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.027477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.027540 (+ 63us) server_negotiation.cc:407] Connection header received
0619 02:49:28.027577 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.027578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.027585 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.027629 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.028054 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.028485 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.029234 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.029465 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.029529 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.031523 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.031525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.031525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.031532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.032813 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.033099 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.033100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.033100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.033150 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.033334 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.033335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.033336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.033461 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.033621 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.033657 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.033730 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:49:28.033787 9836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:49:28 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
Jun 19 02:49:28 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:49:28.040421 9839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.033999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32850)
0619 02:49:28.034041 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:28.034050 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:28.034082 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:28.034429 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:28.034430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:28.034532 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:28.034563 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.034564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.035402 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.035403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.036091 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.036093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.036557 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.036557 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.036599 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.036911 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:28.036915 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:28.037931 (+ 1016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:28.039321 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.039322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.039322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.039723 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.039909 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.039909 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.039910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.039969 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.040151 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:28.040152 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:28.040194 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:49:28.040271 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:49:28.040359 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 02:49:28.040525 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.034243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.19.113.124:46579)
0619 02:49:28.034301 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.034303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.034309 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:49:28.034326 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.034327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.034337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.034388 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.034626 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.035344 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.036163 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.036492 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.036551 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.038000 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.038001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.038001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.038007 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.039251 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.039799 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.039800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.039800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.039855 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.040020 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.040021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.040021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.040103 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.040143 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.040293 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.040421 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":40}
I20260619 02:49:28.041299 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.034611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.19.113.125:38641)
0619 02:49:28.034643 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.034645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.034652 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:49:28.034680 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.034681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.034688 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.034720 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.035256 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.035900 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.036702 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.036905 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.036967 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.038816 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.038817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.038817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.038832 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.040236 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.040622 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.040623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.040624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.040679 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.040864 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.040865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.040865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.040963 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.041108 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.041141 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.041217 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:49:28.041735 9836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe: Bootstrap starting.
I20260619 02:49:28.042495 9836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:28.042729 9836 log.cc:824] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:28.043551 9836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe: No bootstrap required, opened a new log
I20260619 02:49:28.044145 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:28.045248 9836 raft_consensus.cc:348] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } }
I20260619 02:49:28.045408 9836 raft_consensus.cc:729] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c586e8000eec4d639a4a7840e140edbe, State: Initialized, Role: FOLLOWER
I20260619 02:49:28.045482 9836 consensus_queue.cc:260] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } }
I20260619 02:49:28.045711 9842 sys_catalog.cc:455] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.045765 9836 sys_catalog.cc:564] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:28.045795 9842 sys_catalog.cc:458] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:28.044991 9660 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7e4b4aee7ac947499c7b351c620dca31"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826
W20260619 02:49:28.047418 9853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:28.047500 9853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:28.047808 9768 master_runner.cc:186] Error getting master registration for 127.19.113.126:39757: OK, instance_id { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" instance_seqno: 1781837367724005 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:49:28.050540 9768 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:49:28.063812 9686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } attempt: 2
Jun 19 02:49:28 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:49:28.071213 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.064242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.19.113.124:46579)
0619 02:49:28.064307 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.064308 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.064318 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:49:28.064341 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.064342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.064349 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.064425 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.064746 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.065483 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.066440 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.066732 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.066787 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.068935 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.068937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.068937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.068944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.070176 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.070479 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.070480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.070480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.070530 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.070685 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.070686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.070687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.070820 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.070913 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.071044 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.071118 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:49:28.071213 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.064171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32864)
0619 02:49:28.064229 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:28.064239 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:28.064279 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:28.064444 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:28.064444 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:28.064629 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:28.064679 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.064681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.065555 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.065556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.066366 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.066369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.066792 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.066792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.066857 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.067343 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:28.067348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:28.068858 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:28.070252 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.070252 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.070253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.070419 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.070582 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.070583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.070583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.070638 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.070899 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:28.070900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:28.070947 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 02:49:28.071017 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:49:28.071117 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 02:49:28.072937 9686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31: Bootstrap starting.
I20260619 02:49:28.073504 9686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:28.073729 9686 log.cc:824] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:28.074381 9686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31: No bootstrap required, opened a new log
I20260619 02:49:28.075398 9686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } }
I20260619 02:49:28.075505 9686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e4b4aee7ac947499c7b351c620dca31, State: Initialized, Role: FOLLOWER
I20260619 02:49:28.075556 9686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } }
I20260619 02:49:28.075769 9856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.075806 9686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:28.075845 9856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:28.075990 9660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:49:28.076098 9660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:28.076882 9660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } }
I20260619 02:49:28.077534 9810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e4b4aee7ac947499c7b351c620dca31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c586e8000eec4d639a4a7840e140edbe"
I20260619 02:49:28.077644 9810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [term 0 FOLLOWER]: Advancing to term 1
W20260619 02:49:28.077862 9867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:28.077907 9867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:28.078449 9810 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e4b4aee7ac947499c7b351c620dca31 in term 1.
I20260619 02:49:28.078677 9634 leader_election.cc:302] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e4b4aee7ac947499c7b351c620dca31, c586e8000eec4d639a4a7840e140edbe; no voters:
I20260619 02:49:28.078771 9856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:28.078933 9856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [term 1 LEADER]: Becoming Leader. State: Replica: 7e4b4aee7ac947499c7b351c620dca31, State: Running, Role: LEADER
I20260619 02:49:28.079017 9856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } }
I20260619 02:49:28.079047 9660 leader_election.cc:288] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [CANDIDATE]: Term 1 election: Requested vote from peers 7559d21d76d94ee8acde492793bbd3ea (127.19.113.125:38641), c586e8000eec4d639a4a7840e140edbe (127.19.113.124:46579)
I20260619 02:49:28.079413 9856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e4b4aee7ac947499c7b351c620dca31. Latest consensus state: current_term: 1 leader_uuid: "7e4b4aee7ac947499c7b351c620dca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.079480 9618 master_runner.cc:186] Error getting master registration for 127.19.113.125:38641: OK, instance_id { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" instance_seqno: 1781837367865706 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:49:28.081776 9618 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:49:28.079483 9856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:28.082090 9877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:28.082392 9877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:28.089548 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.080396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54832 (local address 127.19.113.126:39757)
0619 02:49:28.080436 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.080438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.080984 (+ 546us) server_negotiation.cc:407] Connection header received
0619 02:49:28.081072 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.081073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.081084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.081111 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.081825 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.082496 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.084807 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.085097 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.085137 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.086594 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.086595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.086595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.086602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.088247 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.088592 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.088593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.088593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.088652 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.088847 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.088848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.088849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.088986 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.089103 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.089363 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.089454 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:49:28.094100 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.080708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.19.113.125:38641)
0619 02:49:28.080744 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.080746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.083004 (+ 2258us) server_negotiation.cc:407] Connection header received
0619 02:49:28.083099 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.083100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.083111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.083138 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.083810 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.084500 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.086392 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.086674 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.086713 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.090942 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.090957 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.090958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.090966 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.092459 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.092980 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.092981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.092981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.093053 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.093344 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.093345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.093346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.093508 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.093593 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.093905 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.093995 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:49:28.095269 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.084599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32880 (local address 127.19.113.124:46579)
0619 02:49:28.084638 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.084640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.085594 (+ 954us) server_negotiation.cc:407] Connection header received
0619 02:49:28.085688 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.085689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.085700 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.085728 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.087309 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.087999 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.089842 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.090130 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.090185 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.092380 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.092381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.092382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.092388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.093861 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.094269 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.094271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.094271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.094342 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.094657 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.094658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.094658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.094782 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.094898 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.095099 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.095178 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:49:28.095889 9810 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [term 1 FOLLOWER]: Refusing update from remote peer 7e4b4aee7ac947499c7b351c620dca31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:28.096227 9856 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:28.097649 9842 sys_catalog.cc:455] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e4b4aee7ac947499c7b351c620dca31. Latest consensus state: current_term: 1 leader_uuid: "7e4b4aee7ac947499c7b351c620dca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.097740 9842 sys_catalog.cc:458] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:28.098065 9761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } attempt: 3
I20260619 02:49:28.098987 9869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e4b4aee7ac947499c7b351c620dca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.099104 9869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:28.099128 9842 sys_catalog.cc:455] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e4b4aee7ac947499c7b351c620dca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.099190 9842 sys_catalog.cc:458] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:28.099967 9880 mvcc.cc:205] Tried to move back new op lower bound from 7298405859717963776 to 7298405859652403200. Current Snapshot: MvccSnapshot[applied={T|T < 7298405859717963776}]
I20260619 02:49:28.100261 9877 catalog_manager.cc:1347] Generated new cluster ID: c7925036b7704b58a067978c1fc60e43
I20260619 02:49:28.100309 9877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:28.101238 9878 mvcc.cc:205] Tried to move back new op lower bound from 7298405859717963776 to 7298405859652403200. Current Snapshot: MvccSnapshot[applied={T|T < 7298405859717963776 or (T in {7298405859717963776})}]
Jun 19 02:49:28 dist-test-slave-zqpb krb5kdc[9605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:49:28.106649 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.098330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:46579 (local address 127.0.0.1:32882)
0619 02:49:28.098370 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:28.098408 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:28.098455 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:28.098618 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:28.098619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:28.098778 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:28.098824 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.098827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.099751 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.099752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.100511 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:28.100514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.101001 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.101002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:28.101055 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.101600 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:28.101606 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:28.103466 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:28.105591 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.105592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.105593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.105808 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.106007 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:28.106008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:28.106008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:28.106070 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:28.106337 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:28.106338 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:28.106400 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 02:49:28.106456 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:49:28.106550 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 02:49:28.107120 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:28.098416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32882 (local address 127.19.113.124:46579)
0619 02:49:28.098450 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.098452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.098460 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:49:28.098515 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.098516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.098525 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.098574 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.098928 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.099664 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.100610 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:28.100898 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:28.100952 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:28.103566 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:28.103568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:28.103568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:28.103575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:28.105501 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.105881 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.105882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.105883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.105943 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:28.106130 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:28.106131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:28.106131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:28.106258 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:28.106307 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:28.106947 (+ 640us) server_negotiation.cc:299] Negotiation successful
0619 02:49:28.107028 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:49:28.109093 9761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea: Bootstrap starting.
I20260619 02:49:28.109849 9761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:28.110143 9761 log.cc:824] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:28.116992 9761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea: No bootstrap required, opened a new log
I20260619 02:49:28.119091 9761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } }
I20260619 02:49:28.119232 9761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7559d21d76d94ee8acde492793bbd3ea, State: Initialized, Role: FOLLOWER
I20260619 02:49:28.119321 9761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } }
I20260619 02:49:28.119653 9761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:28.119715 9735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e4b4aee7ac947499c7b351c620dca31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7559d21d76d94ee8acde492793bbd3ea"
I20260619 02:49:28.119812 9735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:28.120666 9735 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e4b4aee7ac947499c7b351c620dca31 in term 1.
I20260619 02:49:28.120909 9877 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:28.121150 9734 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [term 1 FOLLOWER]: Refusing update from remote peer 7e4b4aee7ac947499c7b351c620dca31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:28.121411 9869 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:28.121651 9877 catalog_manager.cc:1504] Loading token signing keys...
W20260619 02:49:28.127900 9896 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:28.127977 9896 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:28.128159 9885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e4b4aee7ac947499c7b351c620dca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.128243 9885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:28.128289 9885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e4b4aee7ac947499c7b351c620dca31. Latest consensus state: current_term: 1 leader_uuid: "7e4b4aee7ac947499c7b351c620dca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.128332 9885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:28.128381 9885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e4b4aee7ac947499c7b351c620dca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.128423 9885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:28.131822 9898 mvcc.cc:205] Tried to move back new op lower bound from 7298405859804553216 to 7298405859652403200. Current Snapshot: MvccSnapshot[applied={T|T < 7298405859717963776}]
I20260619 02:49:28.133455 9869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e4b4aee7ac947499c7b351c620dca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4b4aee7ac947499c7b351c620dca31" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 39757 } } peers { permanent_uuid: "7559d21d76d94ee8acde492793bbd3ea" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 38641 } } peers { permanent_uuid: "c586e8000eec4d639a4a7840e140edbe" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 46579 } } }
I20260619 02:49:28.133564 9869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:28.133908 9877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e4b4aee7ac947499c7b351c620dca31: Generated new TSK 0
I20260619 02:49:28.134128 9877 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:49:28.160053 9840 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.158442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.19.113.124:46579)
0619 02:49:28.158487 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.158489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.159313 (+ 824us) server_negotiation.cc:407] Connection header received
0619 02:49:28.159534 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.159535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.159545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.159574 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.159987 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32884: BlockingRecv error: recv got EOF from 127.0.0.1:32884 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:49:28.160638 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.158673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49200 (local address 127.19.113.125:38641)
0619 02:49:28.158709 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.158711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.159390 (+ 679us) server_negotiation.cc:407] Connection header received
0619 02:49:28.160212 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.160213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.160223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.160249 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.160579 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49200: BlockingRecv error: recv got EOF from 127.0.0.1:49200 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 02:49:28.161401 9689 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.158844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.19.113.126:39757)
0619 02:49:28.158912 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.158914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.160873 (+ 1959us) server_negotiation.cc:407] Connection header received
0619 02:49:28.161018 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.161019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.161029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.161058 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.161341 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54842: BlockingRecv error: recv got EOF from 127.0.0.1:54842 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 02:49:28.163590 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.162843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.19.113.125:38641)
0619 02:49:28.162903 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.162905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.163108 (+ 203us) server_negotiation.cc:407] Connection header received
0619 02:49:28.163207 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.163208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.163225 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.163252 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.163544 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49210: BlockingRecv error: recv got EOF from 127.0.0.1:49210 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 02:49:28.166464 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.165746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.19.113.125:38641)
0619 02:49:28.165775 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.165777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.166012 (+ 235us) server_negotiation.cc:407] Connection header received
0619 02:49:28.166097 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.166097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.166106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.166133 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.166421 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49212: BlockingRecv error: recv got EOF from 127.0.0.1:49212 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 02:49:28.170166 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.169456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49228 (local address 127.19.113.125:38641)
0619 02:49:28.169486 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.169488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.169695 (+ 207us) server_negotiation.cc:407] Connection header received
0619 02:49:28.169780 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.169781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.169790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.169816 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.170121 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49228: BlockingRecv error: recv got EOF from 127.0.0.1:49228 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260619 02:49:28.176167 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.175285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.19.113.125:38641)
0619 02:49:28.175315 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.175317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.175628 (+ 311us) server_negotiation.cc:407] Connection header received
0619 02:49:28.175716 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.175717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.175726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.175752 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.176122 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49240: BlockingRecv error: recv got EOF from 127.0.0.1:49240 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:49:28.185998 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.185205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49244 (local address 127.19.113.125:38641)
0619 02:49:28.185240 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.185242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.185482 (+ 240us) server_negotiation.cc:407] Connection header received
0619 02:49:28.185574 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.185575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.185585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.185611 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.185947 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49244: BlockingRecv error: recv got EOF from 127.0.0.1:49244 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 02:49:28.204226 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.203281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.19.113.125:38641)
0619 02:49:28.203323 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.203326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.203623 (+ 297us) server_negotiation.cc:407] Connection header received
0619 02:49:28.203726 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.203727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.203739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.203768 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.204168 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49256: BlockingRecv error: recv got EOF from 127.0.0.1:49256 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:49:28.238299 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.237584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.19.113.125:38641)
0619 02:49:28.237629 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.237631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.237738 (+ 107us) server_negotiation.cc:407] Connection header received
0619 02:49:28.237795 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.237796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.237808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.237886 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.238239 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49262: BlockingRecv error: recv got EOF from 127.0.0.1:49262 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:49:28.304029 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.303324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.19.113.125:38641)
0619 02:49:28.303394 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.303396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.303471 (+ 75us) server_negotiation.cc:407] Connection header received
0619 02:49:28.303549 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.303550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.303561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.303600 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.303971 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49264: BlockingRecv error: recv got EOF from 127.0.0.1:49264 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 02:49:28.433844 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.433209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.19.113.125:38641)
0619 02:49:28.433294 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.433296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.433308 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:49:28.433369 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.433370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.433381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.433434 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.433787 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49270: BlockingRecv error: recv got EOF from 127.0.0.1:49270 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 02:49:28.691676 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:28.691071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.19.113.125:38641)
0619 02:49:28.691129 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:28.691131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:28.691146 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:49:28.691226 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:28.691227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:28.691238 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:28.691283 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:28.691629 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49274: BlockingRecv error: recv got EOF from 127.0.0.1:49274 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:49:29.047982 9853 catalog_manager.cc:1259] Loaded cluster ID: c7925036b7704b58a067978c1fc60e43
I20260619 02:49:29.048059 9853 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe: loading cluster ID for follower catalog manager: success
I20260619 02:49:29.048880 9853 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe: acquiring CA information for follower catalog manager: success
I20260619 02:49:29.049161 9853 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c586e8000eec4d639a4a7840e140edbe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:49:29.128407 9896 catalog_manager.cc:1259] Loaded cluster ID: c7925036b7704b58a067978c1fc60e43
I20260619 02:49:29.128479 9896 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea: loading cluster ID for follower catalog manager: success
I20260619 02:49:29.129231 9896 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea: acquiring CA information for follower catalog manager: success
I20260619 02:49:29.129525 9896 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7559d21d76d94ee8acde492793bbd3ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:49:29.207414 9969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:29.204881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49282 (local address 127.19.113.125:38641)
0619 02:49:29.205014 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:29.205015 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:29.205027 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:49:29.205123 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:29.205123 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:29.205180 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:29.205242 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:29.205387 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.205892 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.206693 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.206944 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.207003 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:29.207137 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:29.207239 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:29.207290 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:29.207326 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 02:49:29.207344 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260619 02:49:29.211555 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:29.209207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.19.113.124:46579)
0619 02:49:29.209303 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:29.209305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:29.209315 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:49:29.209400 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:29.209400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:29.209447 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:29.209509 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:29.209613 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.210091 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.210833 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.211056 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.211110 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:29.211253 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:29.211353 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:29.211413 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:29.211454 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 02:49:29.211475 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":47,"threads_started":1}
I20260619 02:49:29.215607 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:29.213095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.19.113.124:46579)
0619 02:49:29.213130 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:29.213132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:29.213326 (+ 194us) server_negotiation.cc:407] Connection header received
0619 02:49:29.213370 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:29.213371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:29.213381 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:29.213423 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:29.213551 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.214234 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.215035 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.215236 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.215295 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:29.215409 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:29.215433 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:29.215475 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:29.215524 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 02:49:29.215541 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:49:29.215975 9969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:29.213055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49290 (local address 127.19.113.125:38641)
0619 02:49:29.213096 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:29.213098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:29.213217 (+ 119us) server_negotiation.cc:407] Connection header received
0619 02:49:29.213289 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:29.213290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:29.213308 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:29.213352 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:29.213503 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.214234 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.215324 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.215589 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.215625 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:29.215765 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:29.215796 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:29.215838 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:29.215889 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:49:29.215908 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:49:29.216253 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:29.213012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.19.113.126:39757)
0619 02:49:29.213334 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:29.213337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:29.213344 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:49:29.213385 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:29.213386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:29.213432 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:29.213501 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:29.213634 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.214122 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.215242 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:29.215550 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:29.215606 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:29.215832 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:29.215932 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:29.215988 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:29.216141 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:49:29.216167 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":43,"threads_started":1}
I20260619 02:49:29.217728 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9618
I20260619 02:49:29.222440 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9693
I20260619 02:49:29.226512 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9768
2026-06-19T02:49:29Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2696 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:49:29 dist-test-slave-zqpb krb5kdc[9986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:49:29 dist-test-slave-zqpb krb5kdc[9986](info): set up 2 sockets
Jun 19 02:49:29 dist-test-slave-zqpb krb5kdc[9986](info): commencing operation
krb5kdc: starting...
W20260619 02:49:30.170948 19909 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/test-user.keytab.
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:49:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:49:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.113.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
Entry for principal kudu/127.19.113.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab.
I20260619 02:49:30.277508 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:41701
--webserver_interface=127.19.113.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:46259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:41701,127.19.113.125:35233,127.19.113.124:39605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:49:30.354223 9999 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:49:30.354429 9999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:30.354444 9999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:30.354463 9999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:30.355602 9999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:30.355636 9999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:30.355649 9999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:30.355660 9999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:30.355670 9999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:30.357081 9999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:41701,127.19.113.125:35233,127.19.113.124:39605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.126.keytab
--principal=kudu/127.19.113.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.126:41701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.113.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:30.357267 9999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:30.357615 9999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:30.362188 10004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:30.362210 10007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:30.362195 10005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:30.362392 9999 server_base.cc:1034] running on GCE node
I20260619 02:49:30.362764 9999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:30.363083 9999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:30.364217 9999 hybrid_clock.cc:648] HybridClock initialized: now 1781837370364206 us; error 30 us; skew 500 ppm
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:30.366495 9999 init.cc:377] Logged in from keytab as kudu/127.19.113.126@KRBTEST.COM (short username kudu)
I20260619 02:49:30.366961 9999 webserver.cc:460] Webserver started at http://127.19.113.126:44673/ using document root <none> and password file <none>
I20260619 02:49:30.367178 9999 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:30.367223 9999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:30.367344 9999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:30.368362 9999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/data/instance:
uuid: "8e5215389d74429b940c714775432170"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "6bfed481bf5856b774eb94f12d1a0f77"
server_key_iv: "697df257fab71baccbdfcadf0c11ba3c"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.368691 9999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/wal/instance:
uuid: "8e5215389d74429b940c714775432170"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "6bfed481bf5856b774eb94f12d1a0f77"
server_key_iv: "697df257fab71baccbdfcadf0c11ba3c"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.369938 9999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:49:30.370608 10014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:30.370748 9999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:30.370834 9999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/wal
uuid: "8e5215389d74429b940c714775432170"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "6bfed481bf5856b774eb94f12d1a0f77"
server_key_iv: "697df257fab71baccbdfcadf0c11ba3c"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.370956 9999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:30.384521 9999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:30.386441 9999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:30.386566 9999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:30.390130 9999 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.126:41701
I20260619 02:49:30.390161 10066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.126:41701 every 8 connection(s)
I20260619 02:49:30.390458 9999 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/data/info.pb
I20260619 02:49:30.391266 10067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:30.391753 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 9999
I20260619 02:49:30.391849 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-0/wal/instance
I20260619 02:49:30.392027 19909 external_mini_cluster.cc:1428] Setting key 41d4feab95727c9d5ec1bedb0730255d
I20260619 02:49:30.393191 10067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.113.125@KRBTEST.COM; defaulting to no policy
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.125@KRBTEST.COM" created.
I20260619 02:49:30.402915 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.393916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:41701 (local address 127.0.0.1:46408)
0619 02:49:30.394075 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.394113 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.394578 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.394820 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.394822 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.395090 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.395234 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.395239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.396725 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.396726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.397384 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.397386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.398014 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.398015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.398064 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.398534 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.398541 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.399822 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.401431 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.401437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.401439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.401709 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.401990 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.401990 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.401991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.402072 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.402379 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.402384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.402441 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.402521 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.402730 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":85,"thread_start_us":73,"threads_started":1}
I20260619 02:49:30.402920 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.393939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46408 (local address 127.19.113.126:41701)
0619 02:49:30.394087 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.394096 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.394200 (+ 104us) server_negotiation.cc:407] Connection header received
0619 02:49:30.394717 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.394719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.394738 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.394810 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.395344 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.396627 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.397465 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.397941 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.398050 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.399901 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.399907 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.399908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.399914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.401337 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.401843 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.401845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.401846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.401925 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.402125 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.402126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.402126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.402321 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.402397 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.402560 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.402662 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260619 02:49:30.403614 10067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:30.403909 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.403796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56158)
0619 02:49:30.403834 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.403884 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 02:49:30.404057 10017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
I20260619 02:49:30.404472 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.404384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56172)
0619 02:49:30.404423 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.404451 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 02:49:30.404569 10067 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:35233: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab.
Entry for principal kudu/127.19.113.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab.
I20260619 02:49:30.415479 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.125:35233
--webserver_interface=127.19.113.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab
--principal=kudu/127.19.113.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:46259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:41701,127.19.113.125:35233,127.19.113.124:39605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:49:30.453825 10067 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } attempt: 1
I20260619 02:49:30.454288 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.454165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56176)
0619 02:49:30.454225 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.454260 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:49:30.454586 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.454506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56186)
0619 02:49:30.454542 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.454567 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 02:49:30.454689 10067 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:35233: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
W20260619 02:49:30.493347 10074 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:49:30.493549 10074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:30.493566 10074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:30.493583 10074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:30.494742 10074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:30.494784 10074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:30.494797 10074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:30.494808 10074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:30.494818 10074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:30.496281 10074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:41701,127.19.113.125:35233,127.19.113.124:39605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.125.keytab
--principal=kudu/127.19.113.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.125:35233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.113.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:30.496513 10074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:30.496717 10074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:30.501223 10080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:30.501292 10082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:30.501338 10074 server_base.cc:1034] running on GCE node
W20260619 02:49:30.501223 10079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:30.501730 10074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:30.502023 10074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:30.503156 10074 hybrid_clock.cc:648] HybridClock initialized: now 1781837370503144 us; error 26 us; skew 500 ppm
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:30.505380 10074 init.cc:377] Logged in from keytab as kudu/127.19.113.125@KRBTEST.COM (short username kudu)
I20260619 02:49:30.505789 10074 webserver.cc:460] Webserver started at http://127.19.113.125:44605/ using document root <none> and password file <none>
I20260619 02:49:30.505997 10074 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:30.506042 10074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:30.506165 10074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:30.507239 10074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/data/instance:
uuid: "a9b93f16edce4d4487cd4c8eddc2a7da"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "095982395f53623fe672885ef6295f5e"
server_key_iv: "8bd4d8d2b2c3ca314150ea03fb06d858"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.507561 10074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/wal/instance:
uuid: "a9b93f16edce4d4487cd4c8eddc2a7da"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "095982395f53623fe672885ef6295f5e"
server_key_iv: "8bd4d8d2b2c3ca314150ea03fb06d858"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.508745 10074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:49:30.509473 10089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:30.509601 10074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:49:30.509680 10074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/wal
uuid: "a9b93f16edce4d4487cd4c8eddc2a7da"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "095982395f53623fe672885ef6295f5e"
server_key_iv: "8bd4d8d2b2c3ca314150ea03fb06d858"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.509737 10074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:30.520255 10074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:30.522185 10074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:30.522310 10074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:30.522889 10067 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } attempt: 2
I20260619 02:49:30.523279 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.523168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56188)
0619 02:49:30.523215 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.523245 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 02:49:30.523608 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.523535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56198)
0619 02:49:30.523570 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.523592 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 02:49:30.523692 10067 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.125:35233: Network error: Client connection negotiation failed: client connection to 127.19.113.125:35233: connect: Connection refused (error 111)
I20260619 02:49:30.526059 10074 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.125:35233
I20260619 02:49:30.526088 10141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.125:35233 every 8 connection(s)
I20260619 02:49:30.526389 10074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/data/info.pb
I20260619 02:49:30.527197 10142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:30.528651 10142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:30.529688 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 10074
I20260619 02:49:30.529780 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-1/wal/instance
I20260619 02:49:30.529940 19909 external_mini_cluster.cc:1428] Setting key 2373a81375794815cc58a274dc037574
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.113.124@KRBTEST.COM; defaulting to no policy
I20260619 02:49:30.536973 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.529282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.19.113.126:41701)
0619 02:49:30.529331 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.529333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.529493 (+ 160us) server_negotiation.cc:407] Connection header received
0619 02:49:30.530028 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.530029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.530043 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.530093 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.530676 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.531189 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.532114 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.532307 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.532368 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.534468 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.534469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.534470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.534477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.535725 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.536080 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.536081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.536082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.536131 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.536356 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.536357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.536358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.536465 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.536518 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.536769 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.536877 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:49:30.537106 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.529248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:41701 (local address 127.0.0.1:46412)
0619 02:49:30.529385 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.529423 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.529935 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.530159 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.530164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.530401 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.530586 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.530591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.531268 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.531269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.532029 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.532032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.532373 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.532374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.532488 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.532911 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.532919 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.534378 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.535799 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.535806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.535808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.536017 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.536184 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.536185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.536185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.536298 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.536517 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.536524 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.536655 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.536742 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.536895 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260619 02:49:30.537626 10142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.113.124@KRBTEST.COM" created.
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:49:30.544780 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.537906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56208)
0619 02:49:30.537946 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.537955 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.537990 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.538210 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.538211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.538318 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.538373 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.538375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.539264 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.539265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.539990 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.539993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.540581 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.540582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.540629 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.540985 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.540989 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.542072 (+ 1083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.543902 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.543903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.543903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.544053 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.544245 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.544245 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.544245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.544287 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.544557 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.544558 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.544592 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.544635 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.544705 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 02:49:30.544900 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.537975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.19.113.125:35233)
0619 02:49:30.538096 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.538100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.538115 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:49:30.538128 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.538130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.538139 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.538188 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.538440 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.539186 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.540066 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.540514 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.540586 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.542154 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.542163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.542163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.542170 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.543803 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.544125 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.544128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.544129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.544191 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.544324 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.544325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.544325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.544499 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.544575 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.544670 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.544764 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260619 02:49:30.546062 10142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:30.546370 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.546272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42360)
0619 02:49:30.546309 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.546353 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 02:49:30.546487 10091 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
I20260619 02:49:30.546885 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.546764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42368)
0619 02:49:30.546798 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.546821 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 02:49:30.546976 10142 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39605: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.113.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab.
Entry for principal kudu/127.19.113.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab.
I20260619 02:49:30.552942 19909 external_mini_cluster.cc:1326] Running /tmp/dist-test-task85DHGI/build/release/bin/kudu
/tmp/dist-test-task85DHGI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.124:39605
--webserver_interface=127.19.113.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab
--principal=kudu/127.19.113.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.113.84:46259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.113.126:41701,127.19.113.125:35233,127.19.113.124:39605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:49:30.596165 10142 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } attempt: 1
I20260619 02:49:30.596664 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.596517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42384)
0619 02:49:30.596584 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.596629 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 02:49:30.596983 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.596900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42388)
0619 02:49:30.596938 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.596963 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 02:49:30.597056 10142 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39605: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
I20260619 02:49:30.614816 10067 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } attempt: 3
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:49:30.621711 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.615220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56216)
0619 02:49:30.615268 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.615280 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.615346 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.615492 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.615493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.615641 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.615707 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.615709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.616592 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.616593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.617117 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.617119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.617582 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.617583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.617638 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.618037 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.618041 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.619170 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.620877 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.620878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.620879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.621037 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.621217 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.621217 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.621217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.621259 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.621483 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.621484 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.621522 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.621566 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.621643 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 02:49:30.621763 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.615284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56216 (local address 127.19.113.125:35233)
0619 02:49:30.615345 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.615347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.615379 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:49:30.615405 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.615406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.615421 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.615467 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.615773 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.616514 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.617196 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.617515 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.617589 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.619252 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.619253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.619254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.619262 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.620778 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.621097 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.621098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.621098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.621160 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.621304 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.621305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.621305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.621431 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.621494 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.621591 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.621669 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:49:30.622239 10067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:30.622540 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.622455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42398)
0619 02:49:30.622494 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.622520 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:49:30.622812 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.622759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42408)
0619 02:49:30.622782 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.622800 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260619 02:49:30.622908 10067 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39605: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
W20260619 02:49:30.632452 10149 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:49:30.632659 10149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:49:30.632681 10149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:49:30.632709 10149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:49:30.633888 10149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:49:30.633929 10149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:49:30.633946 10149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:49:30.633961 10149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:49:30.633987 10149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:49:30.635535 10149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.113.84:46259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.113.126:41701,127.19.113.125:35233,127.19.113.124:39605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/krb5kdc/kudu_127.19.113.124.keytab
--principal=kudu/127.19.113.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.113.124:39605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.113.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:49:30.635808 10149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:49:30.636013 10149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:49:30.640456 10157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:30.640499 10155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:49:30.640456 10154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:49:30.640664 10149 server_base.cc:1034] running on GCE node
I20260619 02:49:30.640920 10149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:49:30.641175 10149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:49:30.642314 10149 hybrid_clock.cc:648] HybridClock initialized: now 1781837370642302 us; error 25 us; skew 500 ppm
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:49:30.644646 10149 init.cc:377] Logged in from keytab as kudu/127.19.113.124@KRBTEST.COM (short username kudu)
I20260619 02:49:30.645045 10149 webserver.cc:460] Webserver started at http://127.19.113.124:34149/ using document root <none> and password file <none>
I20260619 02:49:30.645246 10149 fs_manager.cc:359] Metadata directory not provided
I20260619 02:49:30.645292 10149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:49:30.645421 10149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:49:30.646417 10149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/data/instance:
uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "ba6cd52c0e8a4b675d928d3775fabfa8"
server_key_iv: "a01a3c13741d3359d965b396dd86c7da"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.646735 10149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/wal/instance:
uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "ba6cd52c0e8a4b675d928d3775fabfa8"
server_key_iv: "a01a3c13741d3359d965b396dd86c7da"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.648018 10149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:49:30.648718 10164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:49:30.648885 10149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:49:30.648962 10149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/data,/tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/wal
uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e"
format_stamp: "Formatted at 2026-06-19 02:49:30 on dist-test-slave-zqpb"
server_key: "ba6cd52c0e8a4b675d928d3775fabfa8"
server_key_iv: "a01a3c13741d3359d965b396dd86c7da"
server_key_version: "encryptionkey@0"
I20260619 02:49:30.649051 10149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:49:30.654031 10067 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } attempt: 1
I20260619 02:49:30.654402 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.654283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42410)
0619 02:49:30.654340 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.654368 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:49:30.654687 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.654608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42420)
0619 02:49:30.654642 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.654664 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 02:49:30.654768 10067 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39605: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
I20260619 02:49:30.664913 10149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:49:30.665186 10142 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } attempt: 2
I20260619 02:49:30.665562 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.665455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42432)
0619 02:49:30.665504 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.665536 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:49:30.665877 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.665793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42438)
0619 02:49:30.665835 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.665858 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:49:30.666148 10142 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.113.124:39605: Network error: Client connection negotiation failed: client connection to 127.19.113.124:39605: connect: Connection refused (error 111)
I20260619 02:49:30.667037 10149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:49:30.667158 10149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:49:30.670761 10149 rpc_server.cc:307] RPC server started. Bound to: 127.19.113.124:39605
I20260619 02:49:30.670846 10216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.113.124:39605 every 8 connection(s)
I20260619 02:49:30.671141 10149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/data/info.pb
I20260619 02:49:30.671947 10217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:49:30.673463 10217 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:49:30.677191 19909 external_mini_cluster.cc:1388] Started /tmp/dist-test-task85DHGI/build/release/bin/kudu as pid 10149
I20260619 02:49:30.677294 19909 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task85DHGI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837111403550-19909-0/minicluster-data/master-2/wal/instance
I20260619 02:49:30.677491 19909 external_mini_cluster.cc:1428] Setting key 9046ff0624a0614d77b8a71d5fd09582
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
I20260619 02:49:30.681751 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.674079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.19.113.126:41701)
0619 02:49:30.674157 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.674160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.674317 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:49:30.674771 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.674772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.674781 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.674816 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.675433 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.675889 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.676804 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.676976 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.677018 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.679168 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.679169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.679169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.679176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.680373 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.680788 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.680790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.680790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.680864 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.681116 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.681117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.681117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.681251 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.681315 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.681571 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.681653 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:49:30.681784 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.674063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.126:41701 (local address 127.0.0.1:46418)
0619 02:49:30.674210 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.674245 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.674683 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.674937 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.674941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.675183 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.675368 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.675372 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.675986 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.675987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.676743 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.676745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.677060 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.677061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.677209 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.677656 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.677665 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.679089 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.680438 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.680442 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.680444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.680718 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.680923 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.680924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.680924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.681054 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.681321 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.681326 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.681449 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.681527 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.681635 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260619 02:49:30.682289 10217 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:49:30.685655 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.677817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.19.113.124:39605)
0619 02:49:30.677966 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.677971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.678020 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:49:30.678074 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.678076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.678095 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.678157 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.678735 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.679560 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.680250 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.680609 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.680679 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.683038 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.683044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.683045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.683050 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.684578 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.684920 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.684923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.684923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.684979 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.685148 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.685148 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.685149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.685302 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.685437 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.685482 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.685552 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:49:30.688556 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.682523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.125:35233 (local address 127.0.0.1:56230)
0619 02:49:30.682562 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.682571 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.682605 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.682738 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.682739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.682841 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.682920 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.682923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.683522 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.683523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.684230 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.684233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.684564 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.684565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.684633 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.685007 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.685011 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.686263 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.687661 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.687662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.687662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.687840 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.688018 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.688019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.688019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.688074 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.688279 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.688279 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.688321 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.688379 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.688460 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 02:49:30.688560 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.682589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56230 (local address 127.19.113.125:35233)
0619 02:49:30.682632 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.682634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.682642 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:49:30.682665 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.682665 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.682674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.682725 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.682981 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.683454 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.684306 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.684502 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.684550 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.686334 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.686335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.686335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.686343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.687586 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.687912 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.687913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.687914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.687968 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.688123 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.688124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.688124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.688228 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.688279 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.688407 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.688470 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:49:30.688858 10217 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.126@KRBTEST.COM
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.124@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:49:30.693486 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.686410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.19.113.126:41701)
0619 02:49:30.686457 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.686458 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.686466 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:49:30.686510 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.686511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.686518 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.686557 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.687144 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.687601 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.688240 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.688438 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.688488 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.690564 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.690565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.690565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.690572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.692066 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.692476 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.692478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.692478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.692563 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.692779 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.692780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.692780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.692943 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.693109 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.693305 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.693393 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:49:30.695211 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.689052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42464)
0619 02:49:30.689089 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.689098 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.689129 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.689240 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.689240 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.689339 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.689365 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.689366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.690080 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.690081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.690592 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.690594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.691018 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.691019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.691074 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.691368 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.691372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.692599 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.694145 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.694146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.694146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.694377 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.694586 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.694587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.694587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.694637 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.694916 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.694916 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.694980 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.695041 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.695123 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 02:49:30.695366 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.689100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.19.113.124:39605)
0619 02:49:30.689133 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.689135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.689146 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:49:30.689181 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.689182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.689191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.689240 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.689420 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.690014 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.690674 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.690962 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.691010 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.692671 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.692672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.692672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.692678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.694093 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.694434 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.694434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.694435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.694511 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.694688 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.694689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.694689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.694814 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.694891 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.695113 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.695206 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:49:30.696768 10217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e: Bootstrap starting.
I20260619 02:49:30.697731 10217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:30.698047 10217 log.cc:824] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:30.698961 10217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e: No bootstrap required, opened a new log
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.113.125@KRBTEST.COM
I20260619 02:49:30.700915 10217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } }
I20260619 02:49:30.701066 10217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f65ef71a42f4b8a9ce7df0d676edb6e, State: Initialized, Role: FOLLOWER
I20260619 02:49:30.701155 10217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } }
I20260619 02:49:30.701462 10217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:30.701737 10223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.701826 10223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:30.702832 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.694486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.19.113.125:35233)
0619 02:49:30.694520 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.694522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.694562 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:49:30.694653 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.694653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.694661 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.694699 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.695301 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.695981 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.697233 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.697523 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.697585 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.700327 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.700329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.700329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.700373 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.701824 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.702164 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.702165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.702165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.702219 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.702382 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.702383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.702383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.702504 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.702652 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.702685 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.702749 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 02:49:30.703208 10234 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:30.703262 10234 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:30.704275 19909 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:49:30.703486 10149 master_runner.cc:186] Error getting master registration for 127.19.113.126:41701: OK, instance_id { permanent_uuid: "8e5215389d74429b940c714775432170" instance_seqno: 1781837370389142 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:49:30.706040 10149 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:49:30.705320 10041 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8e5215389d74429b940c714775432170"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46424
I20260619 02:49:30.729914 10067 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } attempt: 2
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.126@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:49:30.737663 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.730329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.19.113.124:39605)
0619 02:49:30.730391 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.730393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.730403 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:49:30.730428 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.730429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.730439 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.730501 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.730774 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.731561 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.732577 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.732880 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.732943 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.734962 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.734964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.734964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.734971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.736436 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.736832 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.736833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.736834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.736900 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.737087 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.737087 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.737088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.737272 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.737333 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.737484 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.737566 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:49:30.737666 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.730233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42466)
0619 02:49:30.730293 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.730306 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.730368 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.730518 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.730519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.730660 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.730705 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.730707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.731656 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.731657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.732490 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.732493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.732954 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.732955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.733009 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.733468 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.733472 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.734862 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.736529 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.736530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.736531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.736755 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.736966 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.736966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.736967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.737030 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.737327 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.737328 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.737389 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.737456 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.737562 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 02:49:30.739225 10067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170: Bootstrap starting.
I20260619 02:49:30.739938 10067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:30.740222 10067 log.cc:824] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:30.740908 10067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170: No bootstrap required, opened a new log
I20260619 02:49:30.741881 10067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } }
I20260619 02:49:30.741989 10067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e5215389d74429b940c714775432170, State: Initialized, Role: FOLLOWER
I20260619 02:49:30.742064 10067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } }
I20260619 02:49:30.742311 10237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.742374 10067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:30.742391 10237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:30.742489 10041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:49:30.742596 10041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:30.743407 10041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } }
I20260619 02:49:30.743999 10191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e5215389d74429b940c714775432170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e"
I20260619 02:49:30.744124 10191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [term 0 FOLLOWER]: Advancing to term 1
W20260619 02:49:30.744204 10248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:30.744251 10248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:30.744932 10191 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e5215389d74429b940c714775432170 in term 1.
I20260619 02:49:30.745268 10016 leader_election.cc:302] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f65ef71a42f4b8a9ce7df0d676edb6e, 8e5215389d74429b940c714775432170; no voters:
I20260619 02:49:30.745394 10237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:49:30.745523 10237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [term 1 LEADER]: Becoming Leader. State: Replica: 8e5215389d74429b940c714775432170, State: Running, Role: LEADER
I20260619 02:49:30.745600 10237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } }
I20260619 02:49:30.745971 10237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e5215389d74429b940c714775432170. Latest consensus state: current_term: 1 leader_uuid: "8e5215389d74429b940c714775432170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.746034 10237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:30.746299 10041 leader_election.cc:288] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [CANDIDATE]: Term 1 election: Requested vote from peers a9b93f16edce4d4487cd4c8eddc2a7da (127.19.113.125:35233), 2f65ef71a42f4b8a9ce7df0d676edb6e (127.19.113.124:39605)
I20260619 02:49:30.747274 10252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:49:30.745236 9999 master_runner.cc:186] Error getting master registration for 127.19.113.125:35233: OK, instance_id { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" instance_seqno: 1781837370525077 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:49:30.747462 9999 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:49:30.747622 10252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:49:30.749605 10191 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [term 1 FOLLOWER]: Refusing update from remote peer 8e5215389d74429b940c714775432170: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:30.750033 10237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:30.751519 10223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e5215389d74429b940c714775432170. Latest consensus state: current_term: 1 leader_uuid: "8e5215389d74429b940c714775432170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.751607 10223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:30.753289 10223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e5215389d74429b940c714775432170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.753378 10223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:30.753717 10237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e5215389d74429b940c714775432170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.753791 10237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:30.753919 10257 mvcc.cc:205] Tried to move back new op lower bound from 7298405870587584512 to 7298405870574612480. Current Snapshot: MvccSnapshot[applied={T|T < 7298405870587584512}]
I20260619 02:49:30.754038 10259 mvcc.cc:205] Tried to move back new op lower bound from 7298405870587584512 to 7298405870574612480. Current Snapshot: MvccSnapshot[applied={T|T < 7298405870587584512}]
I20260619 02:49:30.754927 10252 catalog_manager.cc:1347] Generated new cluster ID: 2cc5e0f6eabc49b2aa964451ed52171f
I20260619 02:49:30.754971 10252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:49:30.757426 10142 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } attempt: 3
I20260619 02:49:30.759704 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.748169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56246 (local address 127.19.113.125:35233)
0619 02:49:30.748218 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.748220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.748598 (+ 378us) server_negotiation.cc:407] Connection header received
0619 02:49:30.748728 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.748729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.748740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.748769 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.750135 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.750861 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.752685 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.752998 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.753043 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.755603 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.755604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.755618 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.755625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.757367 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.758551 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.758552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.758553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.758623 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.758959 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.758960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.758960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.759091 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.759168 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.759529 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.759606 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Jun 19 02:49:30 dist-test-slave-zqpb krb5kdc[9986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837370, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.113.125@KRBTEST.COM for kudu/127.19.113.124@KRBTEST.COM
I20260619 02:49:30.767746 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.752455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.19.113.124:39605)
0619 02:49:30.752515 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.752517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.759179 (+ 6662us) server_negotiation.cc:407] Connection header received
0619 02:49:30.759272 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.759273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.759283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.759309 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.759993 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.760647 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.761555 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.761833 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.761868 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.764554 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.764555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.764556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.764561 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.765998 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.766367 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.766578 (+ 211us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.766578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.766639 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.766864 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.766888 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.766888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.767010 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.767092 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.767565 (+ 473us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.767640 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:49:30.769933 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.757613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.113.124:39605 (local address 127.0.0.1:42482)
0619 02:49:30.757687 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 02:49:30.757696 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:49:30.757726 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:49:30.760876 (+ 3150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:49:30.760877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:49:30.761022 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:49:30.761056 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.761058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.762481 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.762482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.763271 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:49:30.763273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.764099 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.764100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:49:30.764169 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.764530 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:49:30.764535 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:49:30.765947 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:49:30.768635 (+ 2688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.768636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.768636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.768857 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.769149 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:49:30.769150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:49:30.769150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:49:30.769217 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:49:30.769643 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:49:30.769643 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:49:30.769699 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:49:30.769743 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:49:30.769829 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 02:49:30.770557 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.757787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.19.113.124:39605)
0619 02:49:30.760640 (+ 2853us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.760642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.760675 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:49:30.760692 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.760693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.760710 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.761137 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.761197 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.762369 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.763407 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.763967 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.764316 (+ 349us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.766048 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.766049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.766049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.766055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.768535 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.768947 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.768948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.768948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.769060 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.769296 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.769297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.769297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.769545 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.770131 (+ 586us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.770243 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.770376 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2825,"thread_start_us":145,"threads_started":1}
I20260619 02:49:30.772068 10142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da: Bootstrap starting.
I20260619 02:49:30.772799 10142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da: Neither blocks nor log segments found. Creating new log.
I20260619 02:49:30.773113 10142 log.cc:824] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da: Log is configured to *not* fsync() on all Append() calls
I20260619 02:49:30.774118 10142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da: No bootstrap required, opened a new log
I20260619 02:49:30.776139 10142 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } }
I20260619 02:49:30.776275 10142 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b93f16edce4d4487cd4c8eddc2a7da, State: Initialized, Role: FOLLOWER
I20260619 02:49:30.776345 10142 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } }
I20260619 02:49:30.776646 10142 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:49:30.777268 10252 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:49:30.777829 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:30.752133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46432 (local address 127.19.113.126:41701)
0619 02:49:30.752182 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.752184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.759003 (+ 6819us) server_negotiation.cc:407] Connection header received
0619 02:49:30.759086 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.759087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.759095 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.759119 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.759664 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.760357 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.761299 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:30.761576 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:30.761608 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:30.774958 (+ 13350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:49:30.774960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:49:30.774960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:49:30.774968 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:49:30.776487 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.776868 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.776869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.776870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.776927 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:49:30.777116 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:49:30.777117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:49:30.777117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:49:30.777238 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:49:30.777316 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:30.777677 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 02:49:30.777745 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:49:30.778167 10252 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:49:30.777869 10116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e5215389d74429b940c714775432170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da"
I20260619 02:49:30.778007 10115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:49:30.779356 10115 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [term 1 FOLLOWER]: Refusing update from remote peer 8e5215389d74429b940c714775432170: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:49:30.779619 10265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:49:30.780253 10116 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e5215389d74429b940c714775432170 in term 1.
I20260619 02:49:30.779918 10269 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.780512 10269 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:30.781404 10282 mvcc.cc:205] Tried to move back new op lower bound from 7298405870696996864 to 7298405870574612480. Current Snapshot: MvccSnapshot[applied={T|T < 7298405870587584512}]
W20260619 02:49:30.782277 10280 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:49:30.782377 10280 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:49:30.782524 10269 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e5215389d74429b940c714775432170. Latest consensus state: current_term: 1 leader_uuid: "8e5215389d74429b940c714775432170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.782617 10269 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:30.782678 10269 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e5215389d74429b940c714775432170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.782850 10269 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:49:30.789039 10266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e5215389d74429b940c714775432170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5215389d74429b940c714775432170" member_type: VOTER last_known_addr { host: "127.19.113.126" port: 41701 } } peers { permanent_uuid: "a9b93f16edce4d4487cd4c8eddc2a7da" member_type: VOTER last_known_addr { host: "127.19.113.125" port: 35233 } } peers { permanent_uuid: "2f65ef71a42f4b8a9ce7df0d676edb6e" member_type: VOTER last_known_addr { host: "127.19.113.124" port: 39605 } } }
I20260619 02:49:30.789132 10266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170 [sys.catalog]: This master's current role is: LEADER
I20260619 02:49:30.790808 10252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e5215389d74429b940c714775432170: Generated new TSK 0
I20260619 02:49:30.791074 10252 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:49:30.816458 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.814698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56258 (local address 127.19.113.125:35233)
0619 02:49:30.814741 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.814743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.815798 (+ 1055us) server_negotiation.cc:407] Connection header received
0619 02:49:30.815908 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.815909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.815919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.815948 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.816379 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56258: BlockingRecv error: recv got EOF from 127.0.0.1:56258 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:49:30.817414 10071 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.815178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46446 (local address 127.19.113.126:41701)
0619 02:49:30.815219 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.815220 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.816923 (+ 1703us) server_negotiation.cc:407] Connection header received
0619 02:49:30.817019 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.817020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.817030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.817057 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.817354 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46446: BlockingRecv error: recv got EOF from 127.0.0.1:46446 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:49:30.818387 10267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.815499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42494 (local address 127.19.113.124:39605)
0619 02:49:30.815535 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.815537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.817737 (+ 2200us) server_negotiation.cc:407] Connection header received
0619 02:49:30.817907 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.817908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.817919 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.817947 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.818329 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42494: BlockingRecv error: recv got EOF from 127.0.0.1:42494 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:49:30.820744 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.819970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.19.113.125:35233)
0619 02:49:30.820005 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.820007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.820232 (+ 225us) server_negotiation.cc:407] Connection header received
0619 02:49:30.820331 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.820332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.820350 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.820379 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.820697 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56260: BlockingRecv error: recv got EOF from 127.0.0.1:56260 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 02:49:30.823753 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.823028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.19.113.125:35233)
0619 02:49:30.823060 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.823061 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.823295 (+ 234us) server_negotiation.cc:407] Connection header received
0619 02:49:30.823384 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.823385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.823394 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.823421 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.823710 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56276: BlockingRecv error: recv got EOF from 127.0.0.1:56276 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:49:30.827585 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.826836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.19.113.125:35233)
0619 02:49:30.826892 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.826894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.827126 (+ 232us) server_negotiation.cc:407] Connection header received
0619 02:49:30.827213 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.827214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.827224 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.827250 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.827544 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56290: BlockingRecv error: recv got EOF from 127.0.0.1:56290 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:49:30.833375 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.832646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.19.113.125:35233)
0619 02:49:30.832676 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.832678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.832906 (+ 228us) server_negotiation.cc:407] Connection header received
0619 02:49:30.832993 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.832994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.833003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.833030 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.833320 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56300: BlockingRecv error: recv got EOF from 127.0.0.1:56300 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:49:30.843382 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.842523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.19.113.125:35233)
0619 02:49:30.842560 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.842562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.842824 (+ 262us) server_negotiation.cc:407] Connection header received
0619 02:49:30.842952 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.842953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.842964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.842992 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.843322 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56310: BlockingRecv error: recv got EOF from 127.0.0.1:56310 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 02:49:30.861608 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.860726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56312 (local address 127.19.113.125:35233)
0619 02:49:30.860768 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.860771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.861054 (+ 283us) server_negotiation.cc:407] Connection header received
0619 02:49:30.861148 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.861149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.861163 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.861193 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.861547 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56312: BlockingRecv error: recv got EOF from 127.0.0.1:56312 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:49:30.895610 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.894947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.19.113.125:35233)
0619 02:49:30.894995 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.894997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.895131 (+ 134us) server_negotiation.cc:407] Connection header received
0619 02:49:30.895179 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.895180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.895192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.895237 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.895552 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56320: BlockingRecv error: recv got EOF from 127.0.0.1:56320 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 02:49:30.961273 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:30.960616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.19.113.125:35233)
0619 02:49:30.960669 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:30.960671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:30.960795 (+ 124us) server_negotiation.cc:407] Connection header received
0619 02:49:30.960843 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:30.960844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:30.960856 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:30.960903 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:30.961216 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56324: BlockingRecv error: recv got EOF from 127.0.0.1:56324 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260619 02:49:31.091019 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:31.090333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56330 (local address 127.19.113.125:35233)
0619 02:49:31.090401 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:31.090402 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:31.090489 (+ 87us) server_negotiation.cc:407] Connection header received
0619 02:49:31.090552 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:31.090553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:31.090565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:31.090613 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:31.090956 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56330: BlockingRecv error: recv got EOF from 127.0.0.1:56330 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 02:49:31.348686 10146 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:49:31.348113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56334 (local address 127.19.113.125:35233)
0619 02:49:31.348178 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:31.348180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:31.348191 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:49:31.348250 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:31.348251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:31.348261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:31.348308 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:49:31.348625 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56334: BlockingRecv error: recv got EOF from 127.0.0.1:56334 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:49:31.703655 10234 catalog_manager.cc:1259] Loaded cluster ID: 2cc5e0f6eabc49b2aa964451ed52171f
I20260619 02:49:31.703723 10234 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e: loading cluster ID for follower catalog manager: success
I20260619 02:49:31.704500 10234 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e: acquiring CA information for follower catalog manager: success
I20260619 02:49:31.704751 10234 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2f65ef71a42f4b8a9ce7df0d676edb6e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:49:31.782825 10280 catalog_manager.cc:1259] Loaded cluster ID: 2cc5e0f6eabc49b2aa964451ed52171f
I20260619 02:49:31.782936 10280 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da: loading cluster ID for follower catalog manager: success
I20260619 02:49:31.783700 10280 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da: acquiring CA information for follower catalog manager: success
I20260619 02:49:31.783990 10280 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a9b93f16edce4d4487cd4c8eddc2a7da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:49:31.864279 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:31.861766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.19.113.125:35233)
0619 02:49:31.861882 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:31.861884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:31.861912 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:49:31.861972 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:31.861973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:31.862039 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:31.862108 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:31.862260 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.862783 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.863564 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.863754 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.863808 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:31.863958 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:31.864068 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:31.864131 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:31.864168 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 02:49:31.864190 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":48,"threads_started":1}
I20260619 02:49:31.868047 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:31.865881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.19.113.124:39605)
0619 02:49:31.865975 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:31.865977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:31.865985 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:49:31.866021 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:31.866022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:31.866054 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:31.866117 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:31.866194 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.866659 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.867433 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.867632 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.867679 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:31.867777 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:31.867896 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:31.867951 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:31.867967 (+ 16us) server_negotiation.cc:299] Negotiation successful
0619 02:49:31.867983 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":52,"threads_started":1}
I20260619 02:49:31.872216 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:31.869774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.19.113.124:39605)
0619 02:49:31.869863 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:31.869864 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:31.869871 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:49:31.869898 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:31.869899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:31.869909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:31.869957 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:31.870088 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.870804 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.871612 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.871809 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.871861 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:31.871998 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:31.872031 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:31.872078 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:31.872126 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 02:49:31.872147 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:49:31.872591 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:31.869580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.19.113.125:35233)
0619 02:49:31.869617 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:31.869619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:31.869721 (+ 102us) server_negotiation.cc:407] Connection header received
0619 02:49:31.869827 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:31.869828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:31.869838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:31.869865 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:31.870012 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.870760 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.871918 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.872184 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.872220 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:31.872382 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:31.872413 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:31.872456 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:31.872501 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 02:49:31.872523 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:49:31.873742 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:49:31.869781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46462 (local address 127.19.113.126:41701)
0619 02:49:31.869890 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 02:49:31.869891 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:49:31.869898 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:49:31.869935 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:49:31.869936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:49:31.869986 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:49:31.870077 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:49:31.871027 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.871806 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.872990 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:49:31.873212 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:49:31.873265 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:49:31.873443 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:49:31.873522 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:49:31.873571 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:49:31.873627 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 02:49:31.873650 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260619 02:49:31.875334 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 9999
I20260619 02:49:31.880112 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 10074
I20260619 02:49:31.884271 19909 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task85DHGI/build/release/bin/kudu with pid 10149
2026-06-19T02:49:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2657 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5354 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (260484 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST